Forem: George Griffiths

How trip yourself up with React hooks and break production

George Griffiths — Mon, 10 Jan 2022 10:10:34 +0000

React is so ubiquitous now it's pretty difficult to avoid it if you want to be employed and working in Web application development. React hooks in some ways pushed the ecosystem forward but in other ways have made React applications harder to debug and easier to subtly break things without realising, if you stop following the rules.

In this article is a story and some lessons about how I "corrected" some React code to follow the "rules" of hooks and broke the feature because the existing code happened to violate some of the other React "rules", in a way that was not immediately obvious.

Writing about React is scary

Whenever I write a React article I question whether or not it is a good idea to do so because with its popularity can come some unhelpful comments and responses. To get this out of the way early here some replies on some previous comments I've received on React articles:

"Your team just wrote sloppy react code. End of story." - I'd hate to work on a team with this person, jeez.
"React is NOT a framework, it’s a JavaScript library." - I call it a framework just because it annoys people, for some reason, and I will continue to do so.
"This is just an opinion piece" - ????? It's a blog post?

Anyways... to the article, I promise the above is the only sarcastic/cynical part of the post and I think that there is some useful learning in this story to help me grow and hopefully others can find it useful to when working with the React framework.

The scenario

The codebase that I work on is not open source, however the issue that I ran into could be extracted distilled into some example code as the core problems with the code centre around using some of the built in React hooks such as 'useMemo' and handling the mutation of state.

To demonstrate the problem I developed a completely unoriginal Todo-list example, hopefully the mundaneness of the code will help to skip over the boring bits and get to the issues with the code more quickly!

To demonstrate the problem with the code I have developed 3 examples all implementing the same Todo application:

one with the original implementation "before" I refactored and broke it
one with my refactor which broke the code in a specific way
one with the changes I'd probably make in addition to my original refactor to fix the code.

Full code examples will be available at the end of each section.

In order to reproduce the bug I experienced in the other codebase I needed the following:

Some code to add some state from some data from the "server"
React Context API (Technically optional)
use of useMemo to transform some data from the "server"
A button to "save" the state to the server
Some optimisation logic to check if we should submit the changes to the server or not.

Implementation

The implementation of the TodoList isn't too important, I scaffolded it out using vite. The important aspects of the code in this article live inside of a custom Context provider that implements the functionality of the TodoList.

Here is the outline of the top level App component:

import React from "react";
import { AddTodo } from "./AddTodo";
import "./App.css";
import { SaveMessage } from "./SaveMessage";
import { SaveTodos } from "./SaveTodos";
import { TodoProvider } from "./TodoContext";
import { TodoList } from "./TodoList";
function App() {
  return (
    <TodoProvider>
      <div className="App">
        <div className="Flex">
          <h1>Todo list</h1>
          <SaveTodos />
        </div>
        <SaveMessage />
        <AddTodo />
        <div>
          <h2>Items to do</h2>
          <TodoList />
        </div>
      </div>
    </TodoProvider>
  );
}

export default App;

The TodoProvider is where we will focus on this article, it allows me to implement the logic for adding/deleting and saving todo items and share this between components.

Each of the components use aspects of the context provided by TodoProvider, for example, here is how the AddTodo component accesses the functionality to add a todo item:

import { useTodos } from "./TodoContext";

export const AddTodo = () => {
  const { addTodo } = useTodos();

  return (
    <form
      onSubmit={(e) => {
        e.preventDefault();
        const formEntries = new FormData(e.target);
        addTodo(formEntries.get("message"));
      }}
    >
      <input
        className="Input SearchBox"
        name="message"
        placeholder="New item..."
        id="addItem"
        type="text"
      />

      <button className="Button" type="submit">
        <span role="img" aria-label="Add item">
          ➕
        </span>
      </button>
    </form>
  );
};

I'll add in full code sandbox links at the end of each section.

The original implementation

Remember, this code is not the actual code from the codebase that I work on but is an example distilled from the issue that I came across.

The first thing that the TodoProvider does is fetch some data from the "server", in this case I've hardcoded a json file with some todo items in it.

{
  "todos": [
    {
      "id": 1,
      "message": "Go to the supermarket",
      "done": false
    },
    {
      "id": 2,
      "message": "Mow the lawn",
      "done": true
    },
    {
      "id": 3,
      "message": "Clean the kitchen",
      "done": true
    },
    {
      "id": 4,
      "message": "Book restaurant reservation",
      "done": false
    }
  ]
}

Below I setup a context and fetch the JSON. It might seem odd to you that I am setting the API response into useState, this is a little contrived and just for the demo, before I get any "your code sucks comments". In the case of the real codebase this would have been a graphql useQuery hook.

There is a good reason why I am storing the response and not the the TODOs themselves, because I need to do a transform on the data and this is the closest way I could mimic was the original code was doing without introducing some fancy http client hook like react-query.

export const TodoContext = createContext();

export const TodoProvider = ({ children }) => {
  const [apiResponse, setApiResponse] = useState(undefined);
  const [draftTodos, setTodoList] = useState();

  useEffect(() => {
    const fetchTodos = async () => {
      const res = await fetch("./todos.json");
      const response = await res.json();
      setApiResponse(response);
    };
    fetchTodos();
  }, []);

  // Transforming the data, use of useCallback looks odd here...
  // We'll get to that!
  const existingTodos = useCallback(() => {
    const todoMap = new Map();
    apiResponse?.todos.forEach((todo) => {
        todoMap.set(todo.id, todo);
   });

    return todoMap;
  }, [apiResponse]);

  return  return (
    <TodoContext.Provider value={{
        /* code coming soon */
    }}>
    {children}
    </TodoContext.Provider>
}

The second part of the code uses useCallback to create a function that converts the array of items into a map of todo items where the key is the id e.g.

{
    1: {
      "id": 1,
      "message": "Go to the supermarket",
      "done": false
    },
    2: {
      "id": 2,
      "message": "Mow the lawn",
      "done": true
    },
    3: {
      "id": 3,
      "message": "Clean the kitchen",
      "done": true
    },
    4: {
      "id": 4,
      "message": "Book restaurant reservation",
      "done": false
    }
}

If you think the use of useCallback here is strange and useMemo would make sense, we both had the same thought and I wonder if you would end up introducing the same bug that I did by refactoring and correcting the code!

Now we're fetching Todo Items we want a way of adding new items. Before we do that I'm going to introduce a new concept to the above implementation, the notion of "draft todos", these are modified or new todo items which have not been saved back to the server just yet. To make that happen I add in:

export const TodoProvider = ({ children }) => {
  // New line here!
  const [draftTodos, setTodoList] = useState();

  const [apiResponse, setApiResponse] = useState(undefined);
  useEffect(() => {
    const fetchTodos = async () => {
      const res = await fetch("./todos.json");
      const response = await res.json();
      setApiResponse(response);
    };
    fetchTodos();
  }, []);


  const existingTodos = useCallback(() => {
    const todoMap = new Map();
    apiResponse?.todos.forEach((todo) => {
        todoMap.set(todo.id, todo);
   });

    return todoMap;
  }, [apiResponse]);

  // Other new line!
    useEffect(() => {
    // pass in initial items from server
    if (!draftTodos && existingTodos().size) {
      // this () is strange because useCallback was used
      setTodoList(existingTodos());
    }
  }, [existingTodos]);

  return  return (
    <TodoContext.Provider value={{
        todoItems: draftTodos ? Array.from(draftTodos.values()) : [],
    }}>
    {children}
    </TodoContext.Provider>
}

The purpose of the useEffect is so that on initialisation the draft todos equal the existing todo items.

In the new lines, if it wasn't clear before, hopefully it would be now that useCallback here is quite strange indeed because in order to read the existing you need to execute the existingTodos as a function.

The keen eyed here will notice that this means a new map is being created each time you call the function, resetting the data back to the initial state from the json file. This also means the reference to the Map is different each time meaning unless you pass the output of the function around you aren't comparing the same map! This is key to the issue later on when useMemo is used instead!

Adding and removing Todo items

These next two lines are more or less the existing code which was in place to add or remove items, and were exported onto the context.

<TodoContext.Provider
  value={{
    todoItems: draftTodos ? Array.from(draftTodos.values()) : [],
    removeTodo: (id) => {
      if (draftTodos.delete(id)) {
        setTodoList(new Map(draftTodos));
      }
    },
    addTodo: (message) => {
      if (!message) {
        return;
      }
      const todo = {
        // new web api! - Support gradually increasing
        id: crypto.randomUUID(),
        message,
        done: false,
      };
      if (draftTodos.has(todo.id)) return;
      draftTodos.set(todo.id, todo);
      setTodoList(new Map(draftTodos));
    },
  }}
>
  {children}
</TodoContext.Provider>

These code examples also looked a little off to me too, but I couldn't quite understand why or put two and two together, at first. What the code is doing:

Adding or Removing item from todo list
Making a new Map with the entries from the previous map and setting it to the state.

There was in fact a comment next to the new Map() line which mentioned that this is needed in order to cause React to update, but it hadn't quite clicked with me that this was a bit of a problem.

If you hadn't noticed already, the lines above were mutating the previous map in both the add and remove methods, and then making a copy of it, polluting the previous data. Without any other changes this might not be a big deal at all, a bit bad in terms of best practice of not mutating objects in React, especially those used as state, but it works, but now let's take a look at the code for saving the new items.

Saving items to the "server"

The code below is quite contrived and is missing the context as to why it was needed. But what the code does is check if the data had changed before sending it, it was a bit of an optimisation and there were other reasons too, but that's not important, let's take a look.

<TodoContext.Provider
  value={{
    save: () => {
      // contrived code for the demonstration
      // in the real app this was responsible for deciding if a request should be sent to server or not
      const existingTodoKeys = Array.from(existingTodos().keys());
      const draftTodoKeys = Array.from(draftTodos.keys());
      let todosHasChanges = existingTodoKeys.length !== draftTodoKeys.length;
      // now check entries using ids, unless we know they have changed based on length
      // there are better ways of detecting changes but this demonstrates the issue
      if (!todosHasChanges) {
        const existingTodoValues = Array.from(existingTodos().values());
        const draftTodoValues = Array.from(draftTodos.values());
        for (let todoIndex = 0; todoIndex < draftTodoKeys.length; todoIndex++) {
          // now check each entry
          if (
            existingTodoKeys[todoIndex] !== draftTodoKeys[todoIndex] ||
            existingTodoValues[todoIndex].done !==
              draftTodoValues[todoIndex].done
          ) {
            todosHasChanges = true;
            break;
          }
        }
      }

      if (todosHasChanges) {
        // send off request to server
      }
    },
  }}
>
  {children}
</TodoContext.Provider>

Most of the above code doesn't matter but the general idea is that the two maps are being compared to one another. Here again strikes the existingTodos() which as we established before essentially "resets" the Map back to the original data from the server. It is in fact this property of code with the useCallback ensuring that existingTodos() is the original data from the server that makes this code work at all because the add and remove operations mutate the original array. If it wasn't for existingTodos() always fetching the un-mutated data the code would not function!

I suggest that this is an extremely obscure bug that only happens to not presents itself because of the use of useCallback to expose a function negates the mutation operations on the original map, instead of useMemo for example. Sadly, for me, I didn't notice this at the time... and I refactored it.

I've published a code-sandbox with the full code for this example that uses useCallback.

How I broke production

As suggested to earlier, when working in this area of the code I took the chance to tidy up the code to use useMemo instead of useCallback. Unfortunately, this made the code fall over, and to make things even more fun I was going on vacation later that week, luckily someone in my team came in and reverted back to using useCallback, hopefully I'll be able to make the change to make the code less of a trap with the mutation of the Maps when I return, which I'll discuss at the end.

// main change - change to use useMemo which means existingTodos is the todos, not a function.
const existingTodos = useMemo(() => {
  const todoMap = new Map();
  apiResponse?.todos.forEach((todo) => {
    todoMap.set(todo.id, todo);
  });

  return todoMap;
}, [apiResponse]);

useEffect(() => {
  // removed `existingTodos` from being a function
  if (!draftTodos && existingTodos.size) {
    setTodoList(existingTodos);
  }
}, [existingTodos]);

// And inside of the save function, removed existingTodos() as its not a function anymore
{
  save: () => {
    const existingTodoKeys = Array.from(existingTodos.keys());
    /// ...
    const existingTodoValues = Array.from(existingTodos.values());
  };
}

This change got through testing and code review, and was seemingly correct. However, due to the mutating of the original Map in the add/remove functions (which I'd missed), the first time you added or deleted something it would not be tracked as a change, meaning if you only added or removed one item in the list the request would not be sent to the server because the entries in the existingTodos had been mutated making the Maps essentially equal (not in terms of reference but contents, which is what the server cared about).

Here is a code-sandbox with the full code and demo that introduces the bug.

Note that the checking and unchecking of items in the demo does not have an issue with mutating the original Map.

Missed in testing

As noted above the issue where changes are incorrectly not reported only appears for the first change. The reason being that if you make two changes the code happens to work (sort of) because the array set by useMemo in this example never changes after its been mutated the one time, but the code for adding or removing items will stop modifying the original Map after the first mutation, instead it uses the latest draft todos map. Incidentally, the real testing scenario was a bit more complex and involved reordering items, so you might be able to see how this scenario could get missed.

This would be incredibly difficult to unit test as well, with the code being split across contexts and multiple components, unless you were testing the full end to end interactions you might not catch this at all!

My suggested fix

The fix for the code is quite straight-forward and goes back to the original highlighted issue, the add and remove functions mutating the original map, they should not do this. Alternatively before setting the existing entries to the draftTodos variable you could make a copy then, either would work, although I think I prefer avoiding mutations at the point of new data - in the add and remove functions.

removeTodo: (id) => {
  // make a copy first
  const newTodos = new Map(draftTodos);
  if (newTodos.delete(id)) {
    setTodoList(newTodos);
  }
},

        addTodo: (message) => {
          if (!message) {
            return;
          }
          const todo = {
            id: crypto.randomUUID(),
            message,
            done: false,
          };
          if (draftTodos.has(todo.id)) return;
          // make a copy first
          const newTodos = new Map(draftTodos);
          newTodos.set(todo.id, todo);
          setTodoList(new Map(newTodos));
        },

Here is a code-sandbox of what I believe is functional code.

Learnings

I suppose one of the main lessons here is to always be aware of where code could potentially be mutating state, especially before introducing something like a useMemo into the code base.

More importantly, when refactoring code from an uncommon pattern (the useCallback) to something more typical it’s important to fully understand why that code might have happened in the first place, it's likely it might be a workaround for another issue. Another learning on my part is that this would had made good sense to start a dialogue with the person who wrote the original code, although that's easy to say in retrospect, there are many contributors.

Perhaps code comments might have saved this or some more unit tests, it’s hard to say. I can say for sure that we did do lots of testing in this area before shipping the bug to production but I suppose because it’s a bug that only happens on the first mutation and fixes itself if you do subsequent changes that might help to explain why it was missed. It's possible that some end to end test automation of this functionality might have caught this, as it seems likely that a human tester would go for the more complex scenario over a more basic test, e.g. a single mutation which might be present in an automated test.

Variable names can lie, the name of the variable also had the word existing in it in the other codebase too making it seem like this variable reference was representative of the original unmodified state, but due to the mutations it was not. I suppose it's a learning that you cannot always trust that a variable is what is says it is.

Maybe just don’t useMemo or useCallback or things that complicate the code - the work this component was doing to convert an array into a map, and ensure it wasn’t happening every render could well be redundant or overkill, part of my issue with react hooks is they can force you to introduce complexity for something as simple as converting from one data type to another and in doing this you can get struck by subtle bugs such as code mutating objects incorrectly or the incorrect use of react hooks ghastly dependency arrays.

Summary

I suppose some people might read this and have the take that you should simply "Get Gud", but I'd suggest to take a step back and think about how ubiquitous React is and how many developers of differing experience can be working on a code base, I think it’s examples like this which help to show that it’s super easy to break things by modifying to use a different React hook or mutating something that should not have been mutated, and no Typescript can’t save you here, this was a Typescript codebase!

It might also be a fair argument to suggest that the bug presented here is not React specific, but I can say for sure the complexity that React can bring just to avoid rendering, or not rendering, does not help.

If you want to read about another instance of discussing bugs and areas of growth, I wrote about another scenario here from my previous role: Memory leaks and why should you never write such bad code.

Thank you for reading! If you want to read more of my work, please follow me on dev.to or on Twitter @griffadev, or get me a coffee if you feel like it ☕.

Beautiful drag and drop interactions with react hooks

George Griffiths — Mon, 20 Dec 2021 15:09:45 +0000

In this article we'll explore how you could build a drag and drop sortable table using @dnd-kit and react-table.

When looking for drag and drop libraries in React there are a lot of options, some of the popular ones are:

react-sortable-hoc - considered deprecated in favour of dnd-kit.
react-beautiful-dnd
react-dnd
dnd-kit which we're going to explore today.

Many of the libraries out there are great but some come with some large downsides, namely:

Are intended for specific use-cases or are too restrictive.
Have a large dependency tree, for example some bundle in redux!
Have inconsistent browser support.

One common thing also brought up when looking at these libraries is those that opt to "use the platform" more by utilising the HTML drag and drop APIs and those that deliberately choose to not use it to overcome browser inconsistencies and help improve support for a variety of inputs such as touch, mouse and keyboard.

As part of this investigation I seriously considered react-beautiful-dnd, whilst it has great support and demos, I hit a few stumbling blocks along to way. It's footprint was also a hard sell to add into the application for a single use-case. I also encountered this issue whilst I was investigating a different use case where I wanted horizontal and vertical re-ordering in a grid.

After trying out some other libraries I eventually settled on @dnd-kit, which I have to say is absolutely fantastic. It is technically still in beta, but despite this its documentation is quite mature. On the docs site it says the following:

dnd kit is currently in beta. Issues and bugs should be expected for early releases. The core concepts are stable, but some of the APIs may change in the future.

So why is this library great?

It is a lightweight, hooks-based, utility driven library which gives you helpful primitives and allows you to opt in for additional features, therefore making the library only as large as your use case requires it be, depending on what you need.

The easiest way to get started I think is to work through some of the code in some of their code sandbox demos. I'm personally a big fan of the sortable image grid example, try breaking down the code to see how things work.

The first step to using dnd-kit is to install the packages you need. The library is separated into small micro-libraries, which each provide additional utilities.
The minimum you will require is the package @dnd-kit/core (~11KB gzipped).

Some other useful packages with dnd-kit provides:

@dnd-kit/sortable - building blocks to build sortable interfaces .(~3KB gzipped)
@dnd-kit/utilities - dnd-kit utilities to help with integration. (~1.3KB gzipped)
@dnd-kit/modifiers - modifiers to change behavior of drag and drop, for example restricting the drag and drop movement to vertical or horizontal. (~3.4KB gzipped)
@dnd-kit/accessibility - Hooks to assist with accessibility for example screen reader announcements. (~499B gzipped)

Webpack problems

Note, in my use I found that webpack had some issues de-duplicating @dnd-kit/core which is used inside of @dnd-kit/sortable. Aside from adding to bloat, if this dependency is not de-duplicated the DndContext provided by dnd-kit will not work. If you run into this you can force it use the same copy of the core library, using an alias in your webpack.config.js.

  resolve: {
    alias: {
      // without this webpack dupes the dependencies when other dnd-kit libs use it which breaks context
      "@dnd-kit/core": path.resolve(__dirname, "node_modules/@dnd-kit/core"),
    }
  }

Learning dnd-kit

The best way that I can recommend to understand the library before we get to the example using react-table is to take a look at the example I mentioned earlier (sortable image grid example).

One thing to highlight that is important and quite cool about the library is its DragOverlay component, which you can use to render a custom react component to visualise your dragged item differently when it is being dragged, for example adding a drop shadow, which you'll notice in my sortable table example a little later. It also offers you the opportunity to change how the original rendered item is rendered when it is being dragged by using the isDragging property from useSortable (or useDraggable), for example change it's opacity or color to help indicate the drop area to the user, this is also shown in my up-coming example.

If you prefer to learn through videos, whilst learning the API I watched some of this video before getting my hands stuck into the code.

What could be better?

The library has some really fantastic examples of how to use it on its storybook, however they do suffer from the classic problem that storybook often presents, they are great as showcases but not very good at demonstrations of how to adapt the example to use the code for yourself. By that I mean, it's very hard to find and understand the code written, and often suffers from the problem of "too much code re-use for demos", although if i'm honest I think this is a problem with Storybook not the library itself.

For example, it's really hard to un-pick what the code in the example is doing, you end up having to search through github and reverse engineer complex examples and across multiple files, for example:

The other thing that makes me sad is that this great library is locked into the React ecosystem, but I must admit, the ergonomics of using custom hooks here is quite nice.

Using react table

react-table is another lightweight and customisable react library, it allows users to use utilities and hooks to build up tables for their UIs. This description may sounds quite similar to what I described before about dnd-kit where you can use a series of utilities and hooks to implement drag and drop. It is for this reason thatreact-table and dnd-kit fit together really nicely.

To help demonstrate this, I took the "basic" example of react-table from their examples here: https://codesandbox.io/s/github/tannerlinsley/react-table/tree/master/examples/basic and then added in drag and drop support, let's have a look how.

Hooking it up

In order to add support there are roughly these steps:

Install the dnd-kit packages: @dnd-kit/core, @dnd-kit/modifiers, @dnd-kit/sortable, @dnd-kit/utilities.
Wrap the table in DndContext and SortableContext.
Provide an array of string item ids to SortableContext to allow items to be sorted.
Implement drag event handlers for re-ordering items in the array.
Render a row using the useSortable hook passing in the id of the row.
Render a drag handle button component.
Render the CSS transform of the row when dragging.
Rendering the DragOverlay for a custom view of the row when dragging.
When a row is being dragged change it's background color and hide the data, using the isDragging property, to act as a "slot" for the user to drop into.

Here is an implementation of how you could go about doing drag and drop re-orderable rows with dnd-kit and react-table: https://codesandbox.io/s/react-table-drag-and-drop-sort-rows-with-dnd-kit-btpy9

This example isn't really focused on making it look "pretty", it needs lots of styling to make it more production ready but hopefully you get the idea.

One really important thing that is missing in the example is custom screen reader messages for accessibility, you will want to do this to help make sure that your application is inclusive of everybody. There is a great example on the dnd-kit accessibility page.

I'm excited to explore difference uses for this library, once you get the hang of it it's really powerful!

Thank you for reading! If you want to read more of my work, please follow me on Twitter @griffadev, or get me a coffee if you feel like it ☕.

Just a friendly reminder that React isn't really Just JavaScript (Don't nest Functional Components)

George Griffiths — Sat, 06 Nov 2021 16:18:24 +0000

This is my first blog post I've written about React, despite reluctantly using it every day due to the UK frontend job market. I'll probably regret talking about such a popular Framework, oh and it is a Framework 😉.

Recently, I was writing an animated slide-in sidebar component, which could be opened to reveal some additional details for another component in the page. One of my goals was to ensure that the tab and focus order of the page made sense when the details pane was opened, namely, I wanted to be able to "steal" focus on open and restore focus back to the original button on close. For example on button press (with space-bar) you should be able to open the details panel, have the panel but focused and close it again with the space-bar. Focus is then returned back to the original button and you can hit "tab" to move to the next item.

Here is a simplified example of what I was building, with some HTTP Status code kitties, try it out for yourself over in this Code sandbox.

Here is the code for the App, this was hastily written HTML which i'm sure could do with being improved.

import "./styles.css";
import { useState } from "react";
import { Sidebar } from "./Sidebar";
export default function App() {
  const statusCodes = [500, 404, 403, 401, 418, 420, 301, 302, 200, 201, 204];
  const [selectedCode, setSelectedCode] = useState(null);

  const Codes = (
    <ul>
      {statusCodes.map((code) => (
        <li key={code}>
          <button onClick={() => setSelectedCode(code)}>{code}</button>
        </li>
      ))}
    </ul>
  );

  return (
    <div className="App">
      <h1>HTTP Status Cats</h1>
      {Codes}
      <Sidebar
        onClose={() => setSelectedCode(null)}
        ariaLabel={`${selectedCode} status code info`}
        open={Boolean(selectedCode)}
      >
        <h2>{selectedCode}</h2>
        <img
          alt={`Cat demonstrating HTTP status code: ${selectedCode}`}
          src={`https://http.cat/${selectedCode}.jpg`}
        />
      </Sidebar>
    </div>
  );
}

And the sidebar, which is where the "focus stealing/restoring" happens:

import { useEffect, useRef, useState } from "react";

export const Sidebar = ({ ariaLabel, open, onClose, children }) => {
  const [previousFocus, setPreviousFocus] = useState();
  // now focus inside something, for arguments sake, the close button
  const closeBtnRef = useRef(null);
  useEffect(() => {
    if (open) {
      setPreviousFocus(document.activeElement);
      closeBtnRef?.current?.focus();
    }
    // bit of a hack putting aria label in here so triggers if another option selected.
  }, [open, ariaLabel, closeBtnRef]);

  return (
    <aside aria-label={ariaLabel} aria-hidden={open ? "false" : "true"}>
      <button
        disabled={!open}
        ref={closeBtnRef}
        onClick={() => {
          // restore previous focus
          previousFocus?.focus();
          onClose();
        }}
      >
        Close X
      </button>
      {open && children}
    </aside>
  );
};

This code was working correctly, and then I was working on another pr based on a branch with new changes that I pulled in and I noticed that the focus navigation had started failing.

Actually, a unit test, which tested the focus interactions started failing which is actually pretty awesome! React Testing Library moved unit testing forward SO MUCH in comparison to Enzyme it's great. If it could only run by default inside a browser instead of Jest, that would be great. By default I mean, commonplace in the industry, i'm sure it's achievable.

In my example above the new PR had added the equivalent of the Codes variable which is in the above snippet:

const Codes = (
  <ul>
    {statusCodes.map((code) => (
      <li key={code}>
        <button onClick={() => setSelectedCode(code)}>{code}</button>
      </li>
    ))}
  </ul>
);

<h1>HTTP Status Cats</h1>;
{
  Codes;
}

Except, that wasn't what was added, this was:

const Codes = () => (
  <ul>
    {statusCodes.map((code) => (
      <li key={code}>
        <button onClick={() => setSelectedCode(code)}>{code}</button>
      </li>
    ))}
  </ul>
);

<h1>HTTP Status Cats</h1>;
{
  <Codes />;
}

The difference is very subtle, but very important, what had been added by making Codes a function was a functional React component nested inside of another functional component. Remember Codes was a variable inside of App. This is the sort of thing which can easily not get picked up in code review but it breaks a lot of things.

Here is a broken example: https://codesandbox.io/s/http-status-cats-broken-fiu72?file=/src/App.jsx:508-554

What's going on here is React is rendering the contents of the "App" component on each render and because the inner component is not memoized or anyway react is just throwing it in the bin and re-rendering. Internally this will be causing the DOM element to be removed and re-added thus breaking the focus state being returned to the original button.

Seeing this in the code, it was a non obvious fix, especially when reviewing another persons code, it got me thinking on some things:

Would we have even caught this if we didn't have good tests?
Would we have ever found the cause of it months later when the code wasn't as fresh in the mind?
If React is "Just Javascript" how come it so royally breaks one of JavaScript best features of nesting functions and creating closures.
Why isn't this front and centre of React docs and default lint rules?

I did some digging on the last two points:

The only reference to nested functions that I could find in the official documentation comes from the old classic "Rules of Hooks" page: Don’t call Hooks inside loops, conditions, or nested functions, no mention of nested components though.

As an aside, I love the the fact that the mechanism that React is now mostly entirely based on, "hooks", requires a list of rules to help prevent you from writing "Just JavaScript".

As for a lint rules, there does appear to be one which you can turn on in the popular eslint-plugin-react - no-unstable-nested-components, maybe i'll suggest to my team that we do. I can't think of a good reason when you would want to nest functional component, even if you're getting fancy and using useMemo, surely you'd just be better off writing simpler code.

I find it fun to think that such a tiny change in the code that looks so innocent can break quite a lot in the rendering of React components and something I learnt from this is i'm really going to do my best to make sure I write good sets of unit tests that test both "interactions" and "accessibility" as it's so easy for these things to regress!

Thank you for reading! If you want to read more of my work, please follow me on Twitter @griffadev, or get me a coffee if you feel like it ☕.

Building progress indicator cards with a single css property

George Griffiths — Tue, 19 Oct 2021 00:00:00 +0000

Here is a full demo of the css which I'll show you how to build, by learning the structure of the css background shorthand property.

Before today I've always shied away from full understanding all of the shorthand property capability that is baked into the background attribute but today i'm going to change that, let's work it out together, let's deconstruct this bit of css that constructs the progress indicator card together:

.progress-card {
  background: linear-gradient(
        to right,
        #65ad60 0,
        #65ad60 30%,
        #faebd1 30%,
        #faebd1 100%
      ) no-repeat 0 0/100% 10%, #ffdc9b no-repeat 0 0/100%;
}

You might be thinking, why do this with a single css property? You could just do this with multiple css properties or html elements, heck even the built in meter html element. These things are all true however there may be some cases where just manipulating a single css property to create effects like this is handy:

You're in a design system and you can only change the background property.
You're in some system where you are unable to modify the HTML.
It's fun.

Breaking down the css background property

Prior to experimenting to create this effect i'd never really gone into too much detail about the capabilities of the background property in css, but it is really quite fantastic! The fact that you can apply multiple backgrounds and their properties using a single css property is very powerful and not something i'd really appreciated before.

To understand css snippet from above it would probably be a good idea to expand into using the non-shorthand background properties, to help understand and breakdown what's going on above:

.progress-card {
  background: linear-gradient(
        to right,
        #65ad60 0,
        #65ad60 30%,
        #faebd1 30%,
        #faebd1 100%
      ) no-repeat 0 0/100% 10%, #ffdc9b no-repeat 0 0/100%;
}

is equivalent to:

.progress-card {
  background-image: linear-gradient(
    to right,
    #65ad60 0,
    #65ad60 30%,
    #faebd1 30%,
    #faebd1 100%
  );
  background-color: #ffdc9b;
  background-size: 100% 10%, 100%;
  background-repeat: no-repeat;
}

What is going on is that we have two backgrounds:

a linear-gradient which starts off green #65ad60, moving to the right and at 30% turns into light cream #faebd1.
a block sand-like color of #ffdc9b.

Note how we are separating the background with a comma ,. The same is done for background-size. background-repeat has just one value because I want the same value for each background, but I could have done background-repeat: no-repeat, no-repeat.

For the background-size property we first have width, followed by height. So the first background size says a width of 100% and a height of 10% which gives us the height of the progress bar. The no-repeat is important, without it the background would just duplicate itself until it filled 100% of the image, as repeat is the default. It's probably useful to note that you may have multiple background images too, which can be an image url or a gradient.

Converting to shorthand

To convert the background-image, background-color, background-size and background-repeat to the shorthand background it's similar to the long form properties. There is a slightly different syntax for background-size as you also need to account for background-position which is a property which I have omitted from my long hand, it would be something like:

.progress-card {
  background-image: linear-gradient(
    to right,
    #65ad60 0,
    #65ad60 30%,
    #faebd1 30%,
    #faebd1 100%
  );
  background-color: #ffdc9b;
  background-size: 100% 10%, 100%;
  background-repeat: no-repeat;
  /* this is the default */
  background-position: 0 0;
}

In order to specify the background-size when using the background shorthand you must also specify the position, to do this you separate the position and the size with a forward-slash /.

For example, here is the first of the backgrounds:

background: [image] [repeat] [position-x] [position-y] / [width] [height];

.progress-card {
  background: linear-gradient(
      to right,
      #65ad60 0,
      #65ad60 30%,
      #faebd1 30%,
      #faebd1 100%
    ) no-repeat 0 0/100% 10%;
}

Whilst writing this article I came across this handy website that generates the shorthand for you, test it out with the above css, you should get a single background property generated for you!

Internally it looks like this website uses this package: https://github.com/frankmarineau/shrthnd.js.

Hopefully you can now see how we got to the final css, by adding in thebackground-color and position values after a comma ,.

background: linear-gradient(
      to right,
      #65ad60 0,
      #65ad60 30%,
      #faebd1 30%,
      #faebd1 100%
    ) no-repeat 0 0/100% 10%, #ffdc9b no-repeat 0 0/100%;

There is a lot more to the css background property, as always, your best bet for research is MDN.

Spicy CSS Custom properties

Now we've had a play with the background property, let's see if we can make the css a little more useful, by introducing css custom properties we now have the possibility of re-using the same css with different values for percentage "progress", or changing the background colors.

If you are new to CSS Custom properties Kevin Powell does a great introduction over here.

.progress-card {
  width: 30ch;
  padding: 2rem;
  border-radius: 5px;
  border: solid 1px #f1f1f1;
  box-shadow: 0px 4px 6px -1px rgba(0, 0, 0, 0.5), 0px 2px 4px -1px rgba(0, 0, 0, 0.2);
  color: #6c3d3d;
  --progress-total: #faebd1;
  --progress-indicator: #65ad60;
  --progress-indicator-height: 10%;
  --progress-card-background: #ffdc9b;
  background: linear-gradient(
        to right,
        var(--progress-indicator) 0,
        var(--progress-indicator) var(--progress, 30%),
        var(--progress-total) var(--progress, 30%),
        var(--progress-total) 100%
      ) no-repeat 0 0/100% var(--progress-indicator-height), var(
        --progress-card-background
      ) no-repeat 0 0/100%;
}

<div class="card-container">
  <div class="progress-card" style="--progress:40%;">
    <h4>Collect 5 Boar meat</h4>
    <p>In progress: 2 of 5</p>
  </div>

  <div class="progress-card" style="--progress:20%">
    <h4>Collect 5 pieces of Firewood</h4>
    <p>In progress: 1 of 5</p>
  </div>

  <div class="progress-card" style="--progress:86%">
    <h4>Collect 7 Raptor feathers</h4>
    <p>In progress: 6 of 7</p>
  </div>

  <div class="progress-card" style="--progress:60%">
    <h4>Collect five Iron ore</h4>
    <p>In progress: 6 of 10</p>
  </div>
</div>

Hopefully you can see how you can change the look of each "card" just by modifying the css custom property for that card only.

Here is a live demo of the html and css: https://codepen.io/georgegriff/pen/RwZRBeJ

Make it a component

You could next choose to create a re-usable component, for example a Web Component, that could set the css custom properties and add appropriate aria attributes.

It's important to note that this css-based progress indicator is purely decoration, you will want to ensure that users with screen readers are able to understand what the component is conveying, using aria attributes in your HTML, or appropriate text labels.

You can learn about ARIA basics on MDN here.

Thank you for reading! If you want to read more of my work, please follow me on Twitter @griffadev, or get me a coffee if you feel like it ☕.

Scheduled builds a for website on Netlify

George Griffiths — Wed, 23 Jun 2021 00:00:00 +0000

I recently launched a re-write of my brothers Guitar teaching business website: cgguitar.co.uk, during this rewrite I implemented a feature where I wanted to fetch YouTube playlists at build time.

To achieve my goals, I wanted to scheduled builds of my Netlify website periodically.
Netlify doesn't have this feature built in, however what Netlify does have is a 'webhook' which you can call trigger to your build.

In this post I offer two ways to trigger this build, using Github Actions or CircleCI. Using Github Actions can have a significant downside, depending on the use case, continue read to find more about that!

Finding your build hook in Netlify

Login to Netlify and navigate to your site settings and local the "Build & deploy" section, your build hook will be in there.

You can test this out by making a curl request in your terminal, you should see it trigger your Netlify website build:

curl -X POST -d {} https://api.netlify.com/build_hooks/$NETLIFY_BUILD_HOOK_TOKEN

Important: Ensure you keep NETLIFY_BUILD_HOOK_TOKEN secret, otherwise anyone can call your build and potentially cause you to go over your build quota.

Can Github Actions do this?

Github actions let you perform continuous integration in Github, they seem like a perfect fit here...

In your Github repository, in the following folder .github/workflows, you could create:

name: Scheduled build
on:
  schedule:
  - cron: '00 15 * * *'
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Trigger our build webhook on Netlify
      run: curl -s -X POST "https://api.netlify.com/build_hooks/${TOKEN}"
      env:
        TOKEN: ${{ secrets.NETLIFY_BUILD_HOOK_TOKEN }}

The above configuration will run every day at around 15:00.

You will want to secure your build_hook token in a secret on Github.

I did this originally and thought job done right? Well, not quite.

Github Actions are disabled on projects after 60 days if there is no activity on the repository, meaning bye bye scheduled builds.

The above limitation might be okay, depending on your use case, but for my case where I wanted to make sure I was fetching the latest videos from a YouTube playlist, this was no good because the website's code itself may not be updating very often, but there may be new videos added.

Using CircleCI instead

One alternative to Github actions is to use CircleCI to do this instead, it has a generous free tier too so there should be no charge for this.

You can create a configuration like this:

Create a folder in your Git repository called .circleci
Create a file called config.yml

Populate the config.yml with something like the following (you can use a different image if you wish). This will build every day at 3PM.

version: 2

defaults: &defaults
  machine:
    image: circleci/classic:201710-02
  steps:
    - run: curl -X POST -d {} https://api.netlify.com/build_hooks/$NETLIFY_BUILD_HOOK_TOKEN

jobs:
  docker:
    <<: *defaults

workflows:
  version: 2

  autobuild:
    triggers:
      - schedule:
          cron: "0 14 * * *"
          filters:
            branches:
              only:
                - main
    jobs:
      - docker

Now you can create a project in CircleCI and you should be able to test your build.
NETLIFY_BUILD_HOOK_TOKEN will need to be set as an environment variable, in a similar way to Github, in the setting of CircleCI for your project.

If you want to read more of my work, please follow me on Twitter @griffadev, or get me a coffee if you feel like it ☕.

Adding dynamic content from an API to a static website at build time

George Griffiths — Wed, 23 Jun 2021 00:00:00 +0000

I recently launched a re-write of my brothers Guitar teaching business website: cgguitar.co.uk, during this rewrite I had some guiding principles which I believe are best practices when building any website:

Use the right tool for the job. You don't need a metric-ton of JavaScript for most websites minimum user experiences.
Provide a great baseline experience with No JavaScript whatsoever.
Limit the number of calls to external services to keep the page load fast.

In this post I'll describe my approach to getting embedded YouTube playlist content into the website, at build time, reducing the number calls to YouTube client side to only the embedded video and thumbnails, no calls out to the YouTube Data API. In addition to this, i'll show you how you can keep the site up to date with easy to configure cron jobs (scheduled builds).

The feature that I built, that I will explain, is an embedded YouTube playlist component which fetches all the data and stats for YouTube playlists at build time and renders their video metadata/thumbnails directly into the HTML. You can check out the feature live over at https://www.cgguitar.co.uk/videos/#guitar-lessons.

The problem with client side

Calling out to external APIs/services from your client side JavaScript can introduce you many problems, to name a few:

Security - if you want to hide your token or keep it secure you either have to:

Ensure your token only works on your websites domain, but this doesn't stop people using it from outside of a web browser.
Add some complex proxy set up where you hide the token on a server you manage, requires having a server or proxy configuration.

Rate limiting/charges - most APIs have limits to the number of API calls you can make, or will start charging you for usage:

Your website content doesn't scale, each visitor would be using your token to call the external services for every visit.
You could end up incurring accidental charges!

JavaScript needed - In order to show the data you want to show to user, you need to serve JavaScript to your users:

Depending on Network speed or the amount of JavaScript on the page the user has to wait for the JS to download before seeing any content.
A user may choose to disable JavaScript.
JavaScript may fail to load entirely, rendering a useless experience to users.

Moving your calls to external APIs to build time

This is approach is not a silver bullet, not every feature would support this, e.g. if you want to work with user submitted content.
However, if all you are showing is content that changes infrequently, moving the data fetching to build time can be a really great solution.

The static site I built for my brothers' business uses Eleventy, a fantastic static site generator.
I wrote about getting started with 11ty in How I got started with 11ty.

The next section will assume some knowledge about 11ty, or static site generators in general.

11ty has a plugin called @11ty/eleventy-cache-assets which you can use to fetch any data you like.

const Cache = require("@11ty/eleventy-cache-assets");

module.exports = async function() {
  let url = "https://api.github.com/repos/11ty/eleventy";

  /* This returns a promise */
  return Cache(url, {
    duration: "1d", // save for 1 day
    type: "json"    // we’ll parse JSON for you
  });
};

The awesome thing about this plugin is that once the data is fetched it is cached so future local builds do not have to re-fetch data, meaning your builds can remain lightning fast which is a common characteristic of any 11ty project.

Embedding YouTube playlists at build time

For my feature I decided I wanted to be able to pick and choose which YouTube playlists that I wanted to show in the website, it is however possible to fetch all YouTube playlists for an account too. I wanted to be able to choose so that I could add, order and describe new playlists in my CMS (Netlify CMS).

The playlists in the website are defined as markdown in the code in a folder named playlists, Netlify CMS is configured to read these files e.g:

--------
title: Beginner guitar lessons
name: beginner-guitar-lessons
id: PLA0cAQ-2uoeoJoFfUz9oq9qhmlnsjFRhU
--------

The first step to getting my playlists into 11ty is to define them as a collection, to do this inside of the src/playlists folder I create a playlists.json.

{
    "tags": ["playlist"],
    "permalink": false
}

This creates an 11ty collection of all of the playlists, with their "id", "name" and "descriptions".

Inside of my videos page I can then work with this collection in my Nunjucks template:

{%- if collections.playlists %}
    {%- asyncEach collections.playlist in playlists | fetchYouTubePlaylists %}
    {%- include 'partials/video-playlist.njk' %}
    {%- endeach %}
{%- endif %}

If you are unfamiliar with template languages in 11ty you can read about them over here.

I'll show what partials/video-playlist.njk is later on in the article.

fetchYouTubePlaylists is where the magic happens and where we will start to use @11ty/eleventy-cache-assets.
This is an 11ty filter which is defined in my .eleventy.js config file.

  eleventyConfig.addNunjucksAsyncFilter("fetchYouTubePlaylists", async (playlists, callback) => {
    const data = await getPlaylists(playlists);
    callback(null, data);
  })

Let's take a dive a layer deeper: getPlaylists is making a call to getPlaylistItem which is where i'm actually doing the data caching.

module.exports.getPlaylists = async (playlists) => {
    if(!playlists) {
        return [];
    }
    const lists = await Promise.all(playlists.map((async ({id, title, description}) => {
        const content = await getPlaylistItem(id);
        return {
            title,
            id,
            description,
            link: `https://www.youtube.com/playlist?list=${id}`,
            ...(content || {}),
        };
    })));
    return lists;
}

This function is looping through all of my playlists and fetching the items (videos) in that playlist. It is also adding the name, description and direct link to YouTube for the whole playlist.

Now for getPlaylistItem:

const getPlaylistItem = async (playlistId) => {
    const apiUrl = 'https://www.googleapis.com/youtube/v3/playlistItems';
    const maxResults = 20;
    const order = 'viewCount';
    const url = `${apiUrl}?key=${apiKey}&part=${encodeURIComponent('snippet,contentDetails')}&type=video%2C%20playlist&maxResults=${maxResults}&playlistId=${playlistId}&order=${order}`;

    console.log(`Fetching YouTube videos for playlist: ${playlistId}`);
    const videos = await Cache(url, {
        duration: "1d", // 1 day
        type: "json" // also supports "text" or "buffer"
     });

    const videoIds = videos.items.map(({contentDetails}) => contentDetails.videoId);
    const  metaInfo = await fetchMetaInfo(videoIds);
    return {
        videos: await Promise.all(videos.items.map(async ({snippet, contentDetails}) => {
            const  hqThumbnail =  snippet.thumbnails.maxres || snippet.thumbnails.high || snippet.thumbnails.medium || snippet.thumbnails.default;
            const  smallThumbnail = snippet.thumbnails.medium || snippet.thumbnails.default;
            const defaultThumbnail = snippet.thumbnails.high;

            return {
                hqThumbnail,
                smallThumbnail,
                defaultThumbnail,
                channelTitle: snippet.channelTitle,
                channelId: snippet.channelId,
                title: snippet.title,
                id: contentDetails.videoId,
                ...(metaInfo[contentDetails.videoId] || {}),
            }
        })),
        hasMore: Boolean(videos.nextPageToken)
    }
};

The first few things this function does is:

Set base url for YouTube API: https://www.googleapis.com/youtube/v3/playlistItems
Set the max number of items in a playlist to return on a page
Pass in APIKey and build up url in accordance with the API Docs.

You will want to store your API key as an environment variable e.g. const apiKey = process.env.YT_API_KEY;. For production you can add this environment variable where ever you choose to build/host the site e.g. on Netlify.

Next up it fetches some extra metadata. fetchMetaInfo fetches things like view count and likes, this is another API call which we would be concerned about if this was client side, but since it's build time, who cares!
Implementation available on Github.

Finally i'm looping through all the data and returning an array of videos for each playlist and a flag hasMore if the playlist has more than then 20 items shown. In my HTML when I see this flag I add an link out to YouTube to watch the full playlist.

The above code a modified version of the original, where i'm doing a a few extra things you can checkout the full version on Github.

Progressive Enhancement

Now I have the website fetching the external data, let's see how I could approach displaying the content in the HTML.

When designing an dynamic experience its a good idea to think about what is the minimal experience you can provide without needing JavaScript, and build from there.
You could start out very simply and just load a link <a> to the YouTube videos, perhaps the thumbnail could open a tab to YouTube, this needs no JS at all, and is what I did:

{%- if playlist -%}
 {%- set firstVideo = playlist.videos[0] -%}
 {%- set description = playlist.description or (playlist.templateContent | safe) %}
   <youtube-playlist id="{{playlist.title | slug }}">
         <div class="fallback" slot="fallback">
            <div class="img-btn-wrapper">
                <img decoding="async" loading="lazy" width="{{firstVideo.hqThumbnailWidth}}" height="{{firstVideo.hqThumbnaillWdith}}" src="{{firstVideo.hqThumbnailUrl}}" />
            </div>
            <a rel="noopener" title="Play playlist: {{playlist.title}}" class="" target="_blank" href="{{playlist.link}}"></a>
        </div>
        {%- for video in playlist.videos -%}
            <li {{helpers.spread(video, "data-") | safe}}></li>
        {%- endfor -%}
        {%- if playlist.hasMore -%}
        <a slot="more-link" href="{{playlist.link}}">Watch more on YouTube.</a>
        {%- endif -%}
   </youtube-playlist>
{%- endif -%}

You will see that i'm wrapping the whole code in a youtube-playlist Custom Element.
When the component loads without JavaScript it is just a link out to YouTube, which is then upgraded to a full playlist experience. This will disable the default "link" behavior too.

I'm not going to go into the implementation of my Web Component in this post but you can check out the source code on Github. The general idea is to consume <li> list items as child content inside of my <youtube-playlist> and when JavaScript loads move this content in the Shadow DOM, and make them look pretty/interactive.

Here is my full Nunjucks template for my html:

{%- if playlist -%}
 {%- set firstVideo = playlist.videos[0] -%}
 {%- set description = playlist.description or (playlist.templateContent | safe) %}
   <youtube-playlist id="{{playlist.title | slug }}">
        <a slot="heading" href="#{{playlist.title | slug }}"><h2>{{playlist.title | safe}}</h2></a>
        <p slot="description">{{description}}</p>
         <div class="fallback" slot="fallback">
            <div class="img-btn-wrapper">
                <img decoding="async" loading="lazy" width="{{firstVideo.hqThumbnailWidth}}" height="{{firstVideo.hqThumbnaillWdith}}" src="{{firstVideo.hqThumbnailUrl}}" />
                <svg style="pointer-events:none;" class="playbtn" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32">
                        <g transform="translate(-339 -150.484)">
                            <path fill="var(--White, #fff)" d="M-1978.639,24.261h0a1.555,1.555,0,0,1-1.555-1.551V9.291a1.555,1.555,0,0,1,1.555-1.551,1.527,1.527,0,0,1,.748.2l11.355,6.9a1.538,1.538,0,0,1,.793,1.362,1.526,1.526,0,0,1-.793,1.348l-11.355,6.516A1.52,1.52,0,0,1-1978.639,24.261Z" transform="translate(2329 150.484)"/>
                            <path fill="var(--Primary, #000)" d="M16.563.563a16,16,0,1,0,16,16A16,16,0,0,0,16.563.563Zm7.465,17.548L12.672,24.627a1.551,1.551,0,0,1-2.3-1.355V9.853a1.552,1.552,0,0,1,2.3-1.355l11.355,6.9A1.553,1.553,0,0,1,24.027,18.111Z" transform="translate(338.438 149.922)" />
                        </g>
                </svg>
            </div>
            <a rel="noopener" title="Play playlist: {{playlist.title}}" class="" target="_blank" href="{{playlist.link}}"></a>
        </div>
        {%- for video in playlist.videos -%}
            <li {{helpers.spread(video, "data-") | safe}}></li>
        {%- endfor -%}
        {%- if playlist.hasMore -%}
        <a slot="more-link" href="{{playlist.link}}">Watch more on YouTube.</a>
        {%- endif -%}
   </youtube-playlist>
{%- endif -%}

Using Web Components like this is a perfect way of enhancing a base HTML experience with limited JavaScript.

Periodically building your website

In order to keep the YouTube playlists up to date I want to be able to build the website every day on schedule.

There are many options when it comes to periodically building a website, I wrote about my approach to doing this in: Scheduling builds on Netlify. In brief, I opted to use Circle CI to call my Netlify build hook every day at 3 PM. I tried Github Actions but there is a major limitation to using an Action for this use case, which I go into in the linked article.

Summary

I hope this article was helpful and you can see some of the advantages to moving dynamic content that changes infrequently to be rendered at build time.

If you want to read more of my work, please follow me on Twitter @griffadev, or get me a coffee if you feel like it ☕.

Natural language search for blog posts using TensorflowJS

George Griffiths — Thu, 22 Apr 2021 13:37:55 +0000

In this post i'll go into how you can get started using pre-trained Tensorflow models to do Machine learning in the browser, examine some of the potential gotchas, such as not blocking the main thread with custom logic and consider the impact of the size of models on UX.

The demo that I developed as part of this article is a "search engine" using my blog posts as a data set, which I converted into an API, the idea being: can I find blog posts based on a search query by a user, by comparing the similarity of the query with a blog posts title and description?

Search is a solved problem and there are better ways of achieving the same thing, but I created this to learn and to have a bit of fun!

If you want to check out a live demo for what I built in this post, I've hosted it on my website.

Sentence similarity with TensorflowJS

I'm going to explain how this all works with a smaller example rather than the full demo that I linked earlier, but the source code for the example is available on Github, it's the same code, just with things like UI simplified.

First up, let's load in the library we are going to use. We're just going to load them from a CDN, when you're just experimenting, you don't want to be messing around with build processes.

Create a HTML file called index.html, with the following content:

<!DOCTYPE html>
<html lang="en">
  <head>
      <meta charset="UTF-8">
      <meta http-equiv="X-UA-Compatible" content="IE=edge">
      <meta name="viewport" content="width=device-width, initial-scale=1.0">
      <title>Blog post search</title>
      <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@latest"></script>
      <script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/universal-sentence-encoder"></script>
  </head>
  <body>
    <script type="module" src="index.js"></script>
  </body>
</html>

We're loading in two libraries here, the first is TensorflowJS and the second is a the Universal Sentence Encoder model, which uses TensforflowJS, you can read about over here.

If you want to code along, host your files on a local dev server. I personally recommend the Live Server VS Code extension.

Next, create index.js add the following code:

 // IIFE - because no top level await in all browsers at time of writing.
  (async () => {
    // download the model
    const model = await use.load();
    const blogPosts = ["How I got started with 11ty", "Building a responsive, progressively enhanced, masonry layout with only CSS and HTML", "Using the Web Share API and meta tags, for simple native sharing", "Tips for debugging in 11ty"];
    const userQuery = "Sharing to social media";
    // embed the user input and the blog posts using the model -  explained next!
    const blogPostsTensor = await model.embed(blogPosts);
    const userInputTensor = await model.embed([userQuery]);
  })();

In Chrome, and other browsers soon, you won't need to wrap the code in an IIFE because you could use top level await instead.

This code is loading the model, and then passing our userQuery of "Sharing to social media" and our array of blogPosts into the model.
Doing this converts the sentences into vectors (arrays) with 512 entries in the vector for each sentence, this is how the model sees the sentence.
Universal sentence encoder has been trained on a large vocabulary and is encoding the provided data based on the data it saw during training.

To help make this a bit clearer, blogPostsTensor and userInputTensor will be an instance of tensor2d.
These are 2D arrays (on the GPU) with 512 entries in each of the arrays, which represents a provided phase.

// The following are example embedding output of 512 dimensions per sentence
// Embedding for user input: "Sharing to social media"
// userInputTensor = tf.tensor2d([[0.01305108  0.02235125 -0.03263278, ...]])
// Embedding for: I am a sentence for which I would like to get its embedding.
// blogPostsTensor = tf.tensor2d([[0.05833394 -0.0818501   0.06890938, ...], [0.05833394 -0.0818501   0.06890938, ...]])

Next, in order to find potentially good results based our input sentence we need to check how similar our input vector is to the vectors of the blog post titles, we can achieve this by calculating Cosine Similarity between the vectors, which will give us a value between -1 and 1. 1 being most similar and -1 being not very similar at all.

I'm not going to explain the mathematics of cosine similarity, but i've provided an implementation of it.
If you want to know how it works, there are lots of great explanations on YouTube, such as this one..

Define these at the top of your index.js file.

// multiply with value with corresponding value in the other array at the same index, then sum.
const dotProduct = (vector1, vector2) => {
  return vector1.reduce((product, current, index) => {
    product+= current * vector2[index];
    return product;
  }, 0);
};

// square each value in the array and add them all up, then square root.
const vectorMagnitude = (vector) => {
  return Math.sqrt(vector.reduce((sum, current) => {
    sum += current *  current;
    return sum;
  }, 0));
};

const cosineSimilarity = (vector1, vector2) => {
  return dotProduct(vector1, vector2) / (vectorMagnitude(vector1) * vectorMagnitude(vector2));
};

I tried to implement this maths purely in TensorflowJS, so that I could take advantage of the GPU, but after much trial and error, I could not find a solution. If anyone knows how to do this I'd love to hear about it. Doing this calculation myself is performing a large tradeoff of having these calculations happen on the main thread, which can cause bad UX, i'll explain this in more detail towards the end of the post, including ways around this.

Now lets use the functions in our code,

  (async () => {
    // download the model
    const model = await use.load();
    const blogPosts = ["How I got started with 11ty", "Building a responsive, progressively enhanced, masonry layout with only CSS and HTML", "Using the Web Share API and meta tags, for simple native sharing", "Tips for debugging in 11ty"];
    const userQuery = "Sharing to social media";
    // embed the user input and the blog posts using the model -  explained next!
    const blogPostsTensor = await model.embed(blogPosts);
    // wrap the user input in an array so model can work with it
    const userInputTensor = await model.embed([userQuery]);

    // == New code starts here //
    // convert to JS arrays from the tensors
    const inputVector = await userInputTensor.array();
    const dataVector = await blogPostsTensor.array();

    // this is an array of arrays, we only care about one piece of user input, one search query so
    const userQueryVector = inputVector[0];

    // how many results do i want to show
    const MAX_RESULTS = 2;
    // loop through the blog  post data
    const predictions = dataVector.map((dataEntry, dataEntryIndex) => {
        // COSINE SIMILARITY - compare the user input tensor with each blog post.
        const similarity = cosineSimilarity(userQueryVector, dataEntry);
        return {
          similarity,
          result: blogPosts[dataEntryIndex]
        }
        // sort descending
      }).sort((a, b) => b.similarity - a.similarity).slice(0, MAX_RESULTS);

    document.querySelector("#initial-example-results").innerText = JSON.stringify(predictions, null, 2)
  })();

On the last line of the above example we're updating the text of an element with id "initial-example-results", to make this work, let's add the following to your html file, inside the <body> tag.

<p>This will take a few moments for the model to load and run. Query: "Sharing to social media"</p>
<pre id="initial-example-results"></pre>

Here's a link to the code we've built so far: https://codesandbox.io/s/tensorflow-js-hardcoded-blog-search-0q5o9

Turning posts into an API

My blog is written using the static site generator tool Eleventy. If you haven't heard of Eleventy and you're into building fast websites, seriously check it out, it's awesome. I'm not going to explain how Eleventy works, but I wrote a post about how I got started with Eleventy.

To create an API out of my blog posts I generate a JSON file in the form of a JSON Feed, which can be hosted on my server.

Here's my template for my json feed, this template is based on the 11ty base blog. The templating syntax being used is Nunjucks, and comes supported out of the box with Eleventy.

If you are curious and want to check out the source code of my blog it's over here on Github.

---
# Metadata comes from _data/metadata.json
permalink: "{{ metadata.jsonfeed.path | url }}"
eleventyExcludeFromCollections: true
---
{
  "version": "https://jsonfeed.org/version/1",
  "title": "{{ metadata.title }}",
  "home_page_url": "{{ metadata.url }}",
  "feed_url": "{{ metadata.jsonfeed.url }}",
  "description": "{{ metadata.description }}",
  "author": {
    "name": "{{ metadata.author.name }}",
    "url": "{{ metadata.author.url }}"
  },
  "items": [
    {%- for post in collections.posts | reverse %}
    {%- set absolutePostUrl %}{{ post.url | url | absoluteUrl(metadata.url) }}{% endset -%}
    {
      "id": "{{ absolutePostUrl }}",
      "url": "{{ absolutePostUrl }}",
      "title": "{{ post.data.title }}",
      "tags": [
        {%- for tag in helpers.removeCollectionTags(post.data.tags) -%}
          "{{tag}}"
          {%- if not loop.last %}, {%- endif %}
        {%- endfor %}],
      "summary": "{{ post.data.description }}",
      "content_html": {% if post.templateContent %}{{ post.templateContent | dump | safe }}{% else %}""{% endif %},
      "date_published": "{{ post.date | rssDate }}"
    }
    {%- if not loop.last -%}
    ,
    {%- endif -%}
    {%- endfor %}
  ]
}

This template is iterating through my blog posts and populating a JSON array with post data, as well as some other site metadata, ultimately the result is a JSON file which i can request on my server: https://griffa.dev/feed/feed.json.

Now I have an API which I can use in my search, success!

We can now update our code sample to pull data from this api instead of hard-coding it.
Add this function to the top of "index.js".

  const loadBlogPosts = async () => {
    const res = await fetch("https://griffa.dev/feed/feed.json");
    const feed = (await res.json());
    return feed.items.map((item) => {
      return {
        /* search on title and summary */
        searchData: `${item.title} ${item.summary}`,
        title: item.title,
        description: item.summary
      };
    });
  };

Replace the following code:

  const model = await use.load();
  const blogPosts = ["How I got started with 11ty", "Building a responsive, progressively enhanced, masonry layout with only CSS and HTML", "Using the Web Share API and meta tags, for simple native sharing", "Tips for debugging in 11ty"];

with:

  const [model,blogPosts] = await Promise.all([use.load(),loadBlogPosts()]);

Also replace

  const blogPostsTensor = await model.embed(blogPosts);

with:

  const blogPostsTensor = await model.embed(blogPosts.map(({searchData}) => searchData));

Here's a link to the code we've built so far: https://codesandbox.io/s/tensorflow-js-blog-search-3k7x2

ML in the browser, why?

Hopefully the examples so far have made sense, I thought i'd take a moment to talk about some of benefits and tradeoffs of doing Machine learning in the browser with TensorflowJS.

One of the first things you might think of when you think Machine learning in JavaScript is it's slow, well that's where one of the great things about TensorflowJS comes in, it performs all of its expensive calculations on the GPU, under the hood it's utilising WebGL shader programs to achieve this.

Running Machine learning in the browser opens up the possibilities of offering Machine learning in applications without needing to build complex server architectures, or learning another language. It also means that it's possible to provide on-device Machine learning to users, without their data ever hitting a server.

One of the other great things about the JavaScript ecosystem is its ability to not just run in the browser, but on the server too, with NodeJS. TensorflowJS is also available in Node JS, where it can be bound directly to the Tensorflow API, the same API that the python implementations of the library consume. I've considered the possibility of modifying my experiment in this blog post so that when I generate my static site at build time with Eleventy, I could run the model against my data and pre-generate the data for my blog posts, that might be cool.

The final great thing is that it is possible to convert/re-use models created by the other Tensorflow ecosystems (Python etc) so that they run in the browser.

Now for one of the big trade offs, Machine learning models can be large, there is a lot of work going to make these models smaller and smaller, but the model used in this demo for example is approximately 28 MB. To be fair, for a general purpose natural language model, this is quite impressively small. Many of these models, are split into chunks so that the model can be downloaded in parallel, which improves things a bit. This tradeoff might be acceptable if it unlocks the ability to provide a good enough UX, without the need to hit a server, which once the model is downloaded can be lightning fast. The model can only be as fast the end-user machine it's running on, which, especially on mobile, can vary dramatically.

In applications you might be able to do some different things to make this tradeoff worth it, for example:

Enabling good caching headers
Using service workers to background fetch and cache the model, and enable the feature
Allowing users to opt in/out
Offer the feature as a progressive enhancement that enables once downloaded

With the above tradeoffs in mind it might, or might not, make sense to do ML in the browser. Where you need to try and run your models immediately as the site/app loads, or end user device constraints are a problem, maybe server side is the better choice.

When using JavaScript it's always important to not block the main thread, I mentioned above that Tensorflow utilises the GPU for its calculations, but as soon as you stop using its API you're back in the JS main thread, and if you perform expensive calculations there ,you are at risk of providing a bad UX to your users.

The sample in this post is guilty of this, when performing the cosineSimilarity calculations, let's fix it.

Unblocking the main thread

In the browser you can create additional threads called "Workers", these are isolated threads, that do not have access to any DOM APIs, or variables in the main thread.
The only way to communicate between the main thread is via postMessage, which can be cumbersome.

There is an absolutely fantastic library Comlink that makes working with Worker threads basically invisible, it allows you to work with functions as if they were on the main thread, I believe it achieves this using Proxy objects, hiding the need to work with postMessage directly 🎉.

Let's convert our example to use Comlink and move our maths off the main thread.

We're going to import the Tensorflow libraries in our worker instead so your HTML should look like this.

Let's also add in some user input, to make the demo a bit more spicy.

<!DOCTYPE html>
<html lang="en">
  <head>
      <meta charset="UTF-8">
      <meta http-equiv="X-UA-Compatible" content="IE=edge">
      <meta name="viewport" content="width=device-width, initial-scale=1.0">
      <title>Blog post search</title>
  </head>
  <body>
    <script type="module" src="index.js"></script>
    <form id="search">
      <input disabled name="query" type="text"/>
      <button disabled>Search</button>
    </form>
    <pre id="initial-example-results"></pre>
  </body>
</html>

Next up, delete all of the code in "index.js". Now in "index.js" lets add the code to work with our new "worker.js" file and update the UI.

We're going to add all of the same code, except this time, expose a function called "search" which returns our predictions.
There are few other changes too, such as using importScripts to import the libraries into the Worker.

importScripts("https://unpkg.com/comlink/dist/umd/comlink.min.js");
importScripts("https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@latest");
importScripts("https://cdn.jsdelivr.net/npm/@tensorflow-models/universal-sentence-encoder");

let model;
let blogPosts;

const loadBlogPosts = async () => {
  // fetch + cache comparison data
  const res = await fetch("https://griffa.dev/feed/feed.json");
  const feed = await res.json();
  const data = feed.items.map((item) => {
    return {
        searchData: `${item.title} ${item.summary}`,
      title: item.title,
      description: item.summary
    }
  });
  return data;
};
const loadModel = async () => {
  const model = await use.load();
  return model;
};

const load = async () => {
  [model, blogPosts] = await Promise.all([
      loadModel(),
      loadBlogPosts()
  ])
}

//// cosine similarity fns
const dotProduct = (vector1, vector2) => {
  return vector1.reduce((product, current, index) => {
    product+= current * vector2[index];
    return product;
  }, 0)
};

const vectorMagnitude = (vector) => {
  return Math.sqrt(vector.reduce((sum, current) => {
    sum += current *  current;
    return sum;
  }, 0))
}

const cosineSimilarity = (vector1, vector2) => {
  return dotProduct(vector1, vector2) / (vectorMagnitude(vector1) * vectorMagnitude(vector2))
}
////

async function search(userQuery) {
  const blogPostsTensor = await model.embed(blogPosts.map(({searchData}) => searchData));
  const userInputTensor = await model.embed([userQuery]);

  const inputVector = await userInputTensor.array();
  const dataVector = await blogPostsTensor.array();

  /* this is an array of arrays, we only care about one piece of user input, one search query so */
  const userQueryVector = inputVector[0];

  /* how many results do i want to show */
  const MAX_RESULTS = 5;
  /* loop through the blog  post data */
  const predictions = dataVector.map((dataEntry, dataEntryIndex) => {
      /* compare the user input tensor with tensor of a blog post. */
      const similarity = cosineSimilarity(userQueryVector, dataEntry);
      return {
        similarity,
        result: blogPosts[dataEntryIndex]
      }
      /* sort descending */
    }).sort((a, b) => b.similarity - a.similarity).slice(0, MAX_RESULTS);
  return predictions;
}

const SearchService = {
    search,
    load
}
/* expose the SearchService api to comlink */
Comlink.expose(SearchService);

Now, let's use our new SearchService in "index.js".

import * as Comlink from "https://unpkg.com/comlink@4.3.0/dist/esm/comlink.min.mjs";
const worker = new Worker("worker.js");
const SearchService = Comlink.wrap(worker);

(async () => {
  document.querySelector("#initial-example-results").innerText = "Loading model...";
  await SearchService.load();
  document.querySelector('#search input[name="query"]').disabled = false;
  document.querySelector('#search button').disabled = false;
  document.querySelector("#initial-example-results").innerText = "Model loaded, try out some queries e.g. Building a blog with JavaScript";

  document.querySelector("#search").addEventListener("submit", async (e) => {
    e.preventDefault();
    const data = new FormData(e.target);
    const query = data.get("query");
    document.querySelector("#initial-example-results").innerText = "Searching...";
    const predictions = await SearchService.search(query);
    document.querySelector("#initial-example-results").innerText = JSON.stringify(predictions, null, 2);
  })
})();

If you load this demo code up in the browser you should get similar result to before, but with the heavy-work offloaded to a Worker thread.

Here's a live demo project for reference: https://codesandbox.io/s/tensorflow-js-with-web-workers-blog-searching-1shbq

Hopefully you can see from the example how you can offload work into a worker using Comlink, you can also build for production using popular tools such as Rollup, but I won't cover that here.

One of the neat things about using Worker threads is because they don't have access to the DOM you are forced to separate your application logic from your UI, making your code more modular and reusable in the future.

Future thoughts

Type checking global JavaScript libraries in VS Code for when you just want learn and code.

George Griffiths — Sun, 18 Apr 2021 00:00:00 +0000

I've been learning a little bit of artificial intelligence/machine learning in my spare time and I recently started learning a little TensorflowJS. The quickest and simplest way of getting TensorflowJS is a good old script tag. When you are experimenting and prototyping you really don't want to waste time with build tools.

Whilst this tutorial explains setting up basic type-checking for TensorflowJS it should work for any global library that has TypeScript documentation. When building a production project you may want to consider build tools, but for Machine learning experiments, I find you just want to get to the coding using documentation and examples.

This post explains how you can prototype with files from a CDN whilst opting in to certain nice-to-have features of TypeScript that can assist with learning a library, without fully adopting TypeScript.

You can just load TensorflowJS like so:

<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@latest"></script>

Doing so will make the library available on the global window object as tf

e.g.

window.tf // the Tensorflow library

One downside to doing this is that when you are working with Tensorflow you won't get any auto-complete in your IDE/Editor, because there are no import references for VS Code (or other) to analyze, unlike if you were to use ES6 Imports or Common JS require statements.

Let's take a look at how we can make no changes to the code but give you some nice type checking, and the real bonus for learning a library: inline documentation, which is so fantastic.

Whilst this tutorial explains how to enable type-checking and library documentation in VS Code, it may work for other IDEs, the reason VS Code is so great here is its built in TypeScript support.

Type checking on a global library

Create a directory and open it up in VS Code.

If you don't have it yet, you can download VS Code over here.

Create a simple HTML file called index.html with two script imports, one for the library and one for your code.
We won't actually boot up this file on a server and load it in a browser in this tutorial, but i'm putting this in for completeness.

<!DOCTYPE html>
<html lang="en">
  <head>
  </head>
  <body>
    <script src='https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@latest'></script>
    <script src="./index.js"></script>
  </body>
</html>

The reason for creating a separate index.js instead of inline JavaScript in a <script>, is as far as I could find, type checking does not seem to work in HTML files. If there is a way to get it to work easily, I'd love to know!

If you don't have one yet, first initialise an NPM project in your directory, you may be able to do this without an NPM project by using global libraries with -g, I haven't tried this though.

If you don't have NodeJS and NPM installed you can download it here: https://nodejs.org/en/.

Install the TensorflowJS library, we won't actually be using the code in the library, this is for just referencing the types, which come along with the NPM module.

npm i @tensorflow/tfjs

Next up we're going to create a TypeScript declaration file.
In this file we're going to tell TypeScript that the TensorflowJS library will be available on the global Window interface with a variable named tf

Create a file called index.d.ts

interface Global {
    tf: typeof import("@tensorflow/tfjs")
}

interface Window extends Global {
}

Now finally create index.js and open it up.

Inside the file put the following at the top:

/// <reference path="../index.d.ts" />
const tf = window.tf;

If you want find out more about the notation with the triple-slashes, you can find out more in the TypeScript documentation

Now here comes the magic, when typing you will now get auto-complete, for example if you type tf. you should get some suggestions 🥳.

But the really fantastic thing here, and where it can really help if you are learning the library, is if the library author provides good function documentation you have that available right in your editor.

Adding simple Type-checking

You can also add strict type-checking with single comment in a file.

In index.js add the following to the top of your file.

// @ts-check

You should now see that when you code you will get type warning, like so:

Congratulations, you have now adopted TypeScript!
If you wanted to, you could even define your own types in index.d.ts.

If you want to learn more about ways of adopting TypeScript in small ways, this is a really great video that shows the different "degrees" of adopting TypeScript in a project, without needing to go "all in".

Learning TensorflowJS

If you want to learn more about Machine learning/Tensorflow, I got inspired by watching content from Jason Lengstorf from his Learn with Jason series, which I highly recommend (if it wasn't obvious already). One of the truly awesome things about this series is closed captioning is provided, making this content more accessible to everybody 🎉.

At the time of writing there are 3 sessions relating to Machine Learning and TensorflowJS, here is one of them:

I hope this was a good read, if you feel like reading more of my work, please follow me on Twitter @griffadev, or get me a coffee if you feel like it ☕.

Memory leaks and why should you never write such bad code, even for tests

George Griffiths — Thu, 18 Mar 2021 00:00:00 +0000

A project that I work on started showing crashed Pods in our Kubernetes (K8s) cluster, which runs a NodeJS server in a container, it was failing with a dreaded "OOMKilled" (Out of memory killed) error, which sent me down the path of learning about profiling NodeJS applications for memory leaks.

If you don't know much about Kubernetes, that doesn't matter for this article, the fact the NodeJS application is running in K8s is incidental to the tale, and I will only mention it briefly when discussing debugging in Kubernetes.

I learnt a lot through investigating this memory leak, not just about memory profiling, but about the risk of writing "quick and dirty code", even if it's not for production use.

Here is an obfuscated and simplified explanation of the terrible, terrible code that I originally wrote.

The bad code

So, a few months ago I wrote some code some unit tests which exercised some file validation logic. When I wrote this code I needed to get access to the supported file extensions for the validator, for doing some checks, which lived somewhere else, so I very lazily dumped the file extensions onto an object that I was processing in the test, just for test purposes, right? it'll be fine.

My simplified version of the production module:

const supportedValidators = ['name'];
module.exports = {
    validators: () => {
        return supportedValidators.map((validator) => {
            // imports ./validators/name.js what name.js does is not important.
            return require(`./validators/${validator}.js`);
        })
    }
}

What's happening here is a function is exported which exposes other modules via a dynamic require statement. This dynamic require statement is very important to the memory leak and i'll come back to it.

As an aside using require like this in the code is blocking, so it might not be the best thing to use. For more modern code we could use import() in ES Module code.

Now for the bad code, I had in my unit test:

const {validators} = require("./validate");

const badFunction = () => {
    const myValidators = validators();
    myValidators.map((validator) => {
        // mutating someone elses object, yeah i know, this is super bad.
        if(!validator.supportedExtensions) {
            validator.supportedExtensions = [];
        }
        // the code didn't do this exactly this is demo
        validator.supportedExtensions.push(".pdf");
    });
}

This code is terrible for so many reasons, but it was only for unit tests. This code looks weird, I know, but in context of what I used this for in our tests does make sense, I promise, but this simplified sample is to demonstrate the memory leak.

Unfortunately, only for unit tests ended up not being true, and here is my first learning from this experience:

Any code that you write anywhere, including inside of unit tests, could be copied by any well meaning developer adapting the code for their use case.

This happened, and unfortunately the "just for test" code with the supportedExtensions = [] and .push(".pdf") came along for the ride, even though it wasn't needed at all for the production use case.

Why is this a memory leak?

So on the face of it if you were to look at the myValidators.map code it doesn't look like i'm holding any references to the supportedExtensions object once badFunction its finished. But that assumption ignores that the Array, or the contents of it, that validators() supplies could be retained in memory forever.

Objects in Memory are available for Garbage collection when there are no references to the object anymore and nothing is holding on to those objects. This is a massive over-simplification, I'll link a great talk later in this article which describes it much better than I ever could.

Do you remember this line of code?

return require(`./validators/${validator}.js`)

A NodeJS module is being imported into the current scope, the important thing about NodeJs modules though is that:

No matter how many times you import a module you always get the a reference to the same module object, so:

const foo = require("./foo");
const foo2 = require("./foo2")

foo === foo2 // true

This means that even if we lose all references to the imported module, it wont be garbage collected.
In this case, this is extremely problematic because validator.supportedExtensions.push(".pdf"); will add a new entry to an array, that lives on this module object forever, every time the function is called. Imagine if this was on an REST API call (it was). Yikes.

A couple more learnings here:

Mutating objects in an array someone else passed to you is dangerous, you have no idea what references are held to that object, your modifications may never be garbage collected.
If you do have to put bad non-production code in, put a massive comment around it warning future developers, not to copy it.
Avoid changing objects you don't control.

On a somewhat related note to the learnings above, Jake Archibald recently wrote about the risks of calling methods with potentially non-future proof parameters, it's a good read: https://jakearchibald.com/2021/function-callback-risks/.

Memory profiling and finding the problem code

When I first realised that we might have a Memory leak I first wanted to make sure I knew what tools where at my disposal,
I headed to YouTube and found this great video on the topic.

One way to profile for memory leaks in NodeJS is to use the --inspect flag when running your process e.g. node --inspect index.js
This starts a remote debugging session by default on port 9229.
I find that the best way to hook into this is via the Chrome Developer Tools.

If you open the Developer Tools when you have a node --inspect process running you should notice a new Node logo, like the one shown below:

When you activate this mode, a new window will open where one of the tabs in the new window is "Memory".

To try and find the issue I selected the option "Allocation instrumentation timeline", I then proceeded to run our application tests which had caused the original "OOMKilled" on our cluster. After these tests ran I stopped the instrumentation and proceeded to sort through the results.
I found that sorting by size and then searching the largest allocated objects helped me find the problem.

Sure enough after filtering through a lot of noise I found something like this:

Luckily for me, I know our codebase quite well and was able to identify the problem area based on the variable name of the large array and also the array contents, which is where to my shame I found my awful code being used inside of an API call.

Memory profile in Kubernetes

Memory profiling in Kubernetes is quite similar to when you are running locally, you need to end up with a server on your machine exposing a debugging session.

Recommendations:

Scale your deployment down to 1 replica.
Edit your deployment so your NodeJS server sets the --inspect flag
Disable liveness and readiness probes for the container, otherwise K8s may kill your session whilst debugging.
Increase your memory limits and requests, profiling can take up to 2x more memory than usual.
Locate the pod you want to debug and run kubectl port-forward pod-name 9229 this will result in the same debugging session as earlier running on your machine, forwarded from the K8s cluster.

Reproducing the memory leak with demo code

If you want to have a play reproducing the memory leak you could do this:

Create the following folder and file: validators/name.js Can be empty module exports:

module.exports = {

}

Create validate.js

const supportedValidators = ['name'];
module.exports = {
    validators: () => {
        return supportedValidators.map((validator) => {
            return require(`./validators/${validator}.js`);
        })
    }
}

Create bad-code.js

const {validators} = require("./validate");

const badFunction = () => {
    const myValidators = validators();
    myValidators.map((validator) => {
        if(!validator.supportedExtensions) {
            validator.supportedExtensions = [];
        }
        // the code didnt do this exactly this is demo
        validator.supportedExtensions.push(".pdf");
    });
}

let index = 0;

setInterval(() => {
    // even though theres no references to myValidators array
    // there is a memory leak with the .push
    badFunction();
    index++;
    console.log(`Running bad code cycle: ${index}`);
},0)

Run node --inspect bad-code.js You will need NodeJS installed https://nodejs.org.
Open Chrome developer tools, you should see the NodeJS logo which says "Open dedicated DevTools for NodeJS"
Run the profiling techniques as described in the previous section.

Learnings

I learnt a lot through this experience.

Avoid writing sloppy code, even if it is non-production.
Mutating data that is passed to you is dangerous, you have no idea what references are held to that object, your modifications may never be garbage collected.
How to Memory profile in NodeJS

I hope this was a good read, if only to laugh at my terrible code, if you feel like reading more of my work, please follow me on Twitter @griffadev, or get me a coffee if you feel like it ☕.

Building a responsive, progressively enhanced, masonry layout with only CSS and HTML

George Griffiths — Wed, 13 Jan 2021 16:50:55 +0000

A few days ago I found that I had talked myself into re-designing a website. The website is for my brothers' guitar teaching business which I had previously worked a few years ago, when I was a student. (if you click that link and view it before the rework, yeah I know the site needs work, that's what i'm doing here).

As part of the redesign I was trying to think of ways to lay out testimonials from students, which may have varying length/content, I stumbled onto the idea of using a masonry layout (think bricks, think Pinterest).

There are many ways to go about building out a Masonry layout, this CSS tricks article lays out a few of the options, Approaches for a CSS Masonry Layout.

When looking into the solution I knew I wanted to adhere to a couple of rules:

No committing the huge sin of using JavaScript for layout.
It's okay if the experience upgrades or downgrades based on browser support.

What are we building

Masonry layout using grid-template-rows: masonry
CSS Columns as a fallback

This article will explain how to build out the following responsive masonry cards layout with only HTML and CSS.
This final demo uses some features only available in Firefox, behind a flag, which is detailed in the article.

Here is the full finished demo code:

You'd think CSS Grid could do it

CSS Grid is amazing, and you'd think it would have shipped with a simple way of doing masonry layouts, the initial versions of the spec however, did not ship anything to really help out with these layouts.

But fear not, Firefox has something cooking!

Before I continue, the specification we're going to look into is experimental, and at the time of writing only is available in Firefox behind a flag, however, this article will outline how you could use the feature today and fallback to another approach for browsers without support.

Enabling Masonry layout in Firefox

It's January 2021, i'm using Firefox 84.0.2, depending on how the spec has progressed, you may not need to enable this flag, it may even be available in other browsers, this link may have more information.

To enable in Firefox:

type about:config into your address bar
Accept the warnings
In the search box find: layout.css.grid-template-masonry-value.enabled and enable it.

Over on MDN you can find this article: CSS Grid Layout > Masonry Layout

It details, an example very similar to this:

With this small amount of CSS we can get a layout which is almost what we want already, exciting! It has hardcoded sizing, lets improve it, and enable responsive masonry layouts,

Building out our example

Let's start with a fresh example, starting with a little html and "lorem ipsum" to generate some "bricks" for masonry layout.

To see the effect you will need to be view these examples in a browser that supports masonry layouts (at the time of writing Firefox behind a flag).

This code will result in a masonry style layout, which automatically reduces the number of columns as the screen shrinks down, until you reach a single column.

If you run this example you should notice that there is no clean edge on the bottom, it has a wonky bottom.

No one likes a ~~soggy~~ wonky bottom.

The masonry specification also allows for a new property: align-tracks, which you can set to stretch to fix this problem.

align-tracks also has some other support modes, beware of potential issues with align-tracks if you want a masonry layout where a "brick" can span multiple columns, hopefully these things will get ironed out as the spec progresses.

.masonry {
      display: grid;
      gap: 1rem;
      grid-template-rows: masonry;
      grid-template-columns: repeat(auto-fill, minmax(300px, 1fr));
      /* no more wonky bottom */
      align-tracks: stretch;
}

/* some styling not important */
.brick {
    color: var(--Text, #F0F0F0);
    background: var(--Primary, #74baff);
    padding:1rem;
}

And now as if by magic, the bottom edge is now straight.

What about a fallback

Okay, so now we made something work in a single browser only, and it's not even shipped there, not ideal.
Let's try to achieve something similar, this time with CSS columns.

I'd never used CSS columns before looking into them for implementing this masonry layout, they are actually quite clever, and they can made to be responsive, without media queries, which we will take a look at now.

For this example, i'm going to start to make use of CSS Custom Properties which will become extremely useful once we enable our progressive enhancement, we can use these variables to share sizes and other config.

For the css we're going to make use of some column properties:

    column-gap: 1rem;
    column-fill: initial;
    column-width: 300px;

Setting a column-width will set a minimum width of a column before columns need to shrink down, meaning on smaller devices we can get a single column, the same as with our CSS grid implementation.

If you want to set a specific number of columns you can use the column-count property. You could adjust the number of columns using CSS media queries too, if you wanted to:

Optional example:

/* larger */
@media (min-width: 1024px) {
  .masonry {
    column-count: 4;
  }
}

/* medium */
@media (max-width: 1023px) and (min-width: 768px) {
  .masonry {
    column-count: 3;
  }
}

/* small, anything smaller will be 1 column by default */
@media (max-width: 767px) and (min-width: 540px) {
  .masonry {
    column-count: 2;
  }
}

We're not going to do that, we're just going to make use of column-width and allow our columns to grow up to a maximum card width.

Same html as before, with some changes to the css:

The result is very similar, however, there are two main differences:

The order of items goes down the columns as opposed to across the rows with css grid. Columns: ⬇️ Grid: ➡️
We have the wonky bottom back (there is no easy fix with css columns)

The first difference may be a deal breaker, depending on the use case, for me, order didn't really matter very much.

Progressive enhancement

Now we've seen how to implement a masonry layout in the future and with a fallback, lets put it all together with the magical @supports keyword in css.

@supports(grid-template-rows: masonry) {
    .masonry {
      display: grid;
      gap: var(--masonry-gap);
      grid-template-rows: masonry;
      grid-template-columns: repeat(auto-fill, minmax(var(--masonry-brick-width), 1fr));
      align-tracks: stretch;
    }

    .masonry > * {
        /* use this to reset the margin that the column variant set */
        margin-bottom: initial;
    }
}

Again, with no changes to the HTML again, we're going to implement an @supports just for the masonry layout using css grid:

.masonry {
    --masonry-gap: 1rem;
    --masonry-brick-width: 300px;
    column-gap: var(--masonry-gap);
    column-fill: initial;
    column-width: var(--masonry-brick-width);
}

.masonry > * {
    break-inside: avoid;
    margin-bottom: var(--masonry-gap);
}

@supports(grid-template-rows: masonry) {
    .masonry {
      display: grid;
      gap: var(--masonry-gap);
      grid-template-rows: masonry;
      grid-template-columns: repeat(auto-fill, minmax(var(--masonry-brick-width), 1fr));
      align-tracks: stretch;
    }

    .masonry > * {
        margin-bottom: initial;
    }
}

/* some styling not important */
.brick {
    color: var(--Text, #F0F0F0);
    background: var(--Primary, #74baff);
    padding:1rem;
}

Now with this example, depending on what your browser supports, you can either get the CSS Grid or the CSS Columns version.

As long as the grid-template-rows: masonry doesn't change between now and more browsers shipping, we have just written some CSS that will automatically upgrade itself over time, fixing the technical debt/UX "bug" of a wonky bottom and vertical ordering instead of horizontal.

Bonus round

In my implementation I was toying around with having different cards backgrounds every nth "brick" in the masonry layout.

.brick:nth-child(4n - 7) {
    background: #5A363A;
}

.brick:nth-child(4n - 6) {
    background: #82212C;
}

.brick:nth-child(4n - 5) {
    background: #3A3E41;
}

.brick:nth-child(4n - 4) {
    background: #292A2B;
}

There are some really neat tricks you can do with the nth-child property in CSS, some useful background reading if you are new to this:

Here's the full demo:

Thanks for reading all the way through, I hope this was useful and one day soonish we will be able to easily deal with wonky bottoms in just CSS.

Setting up ESLint to work with new or proposed JavaScript features such as private class fields.

George Griffiths — Tue, 01 Dec 2020 19:35:59 +0000

Some members in my team this week wanted to make use of Private class fields in a NodeJS server. This proposal is currently shipped in Chrome, Edge, Firefox and NodeJS, with Safari notably absent. In this instance, we wanted to get them working for a backend server application, so support since Node 12, we're good to go, or so I thought, turns out linters aren't always here to save you time.

I summed my feelings on the whole process of figuring this out on Twitter.

// Detect dark theme var iframe = document.getElementById('tweet-1333779719074955264-989'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1333779719074955264&theme=dark" }

Feel free if you want to skip ahead past the story, and to head right to Configuring ESLint.

For this article, i'll be using this code example of using Private class fields, the code used is irrelevant.

export class Animal {
    // this is a private class field!
    #noise = '';

    constructor(noise) {
        this.#noise = noise;
    }

    makeNoise() {
        console.log(this.#noise);
    }
}

The first issue we hit when writing this new code, was of course, the linter started failing, so off to Google we went!

Struggling to finding a solution

A quick search for: eslint private class fields you will most likely end up in this Stack Overflow issue.
It will tell you that ESLint does not support experimental stage 3 features, which is indeed correct, and to:

npm install eslint babel-eslint --save-dev

and to update your ESLint config file over to use:

  "parser": "babel-eslint",

Sadly, it seems this is not an entire solution, it seems to make a couple of assumptions:

You have babel-core installed
You have a babel configuration file set up that knows how to transform code with a preset.
Its possible that when the answer was posted babel-eslint did indeed solve al the problems.

If you are in a NodeJS server module, a lot of these assumptions are probably not met.

If you are a developer that has never had to use Babel because you work on the backend or on a build-less frontend, all this stuff can get daunting very fast.

Additionally, it seems since this answer was posted, things have moved on and the recommended parser now lives at:

    "parser": "@babel/eslint-parser",

The ESLint website does have some information about the Past, Present and Future of the babel-eslint over on its website.

Finding this information out was a bit of an adventure, and even on the official babel or ESLint website, it's super unclear that you need to set up a babel config, and then still, what to put in it. I'm pretty sure the only reason I managed to figure it out in the end was because i'm familiar with the mess that is configuring Webpack, Jest and Babel.

Configuring ESLint

Let's get our new syntax working!

First off, lets do the npm install dance:

npm i eslint @babel/core @babel/eslint-parser @babel/preset-env -D

It's nice to set up a linter task in your package json so you can run npm run lint

  "scripts": {
    "lint": "eslint ./"
  },

I'm opting to use @babel/preset-env because it has an easy way to enabled proposals that are shipped in Browsers/Node. Other presets/plugins are available.

Next we need to construct an .eslintrc file.
You can generate one using: ./node_modules/.bin/eslint --init or just copy this starter:

{
    "env": {
        "browser": true,
        "es2021": true,
        "node": true
    },
    "extends": "eslint:recommended",
    "parser": "@babel/eslint-parser",
    "parserOptions": {
        "ecmaVersion": 12,
        "sourceType": "module"
    },
    "rules": {
    }
}

Now if you run npm run lint You will hit the following error:

/path/to/code/Animal.js
  0:0  error  Parsing error: No Babel config file detected for /path/to/code/Animal.js. Either disable config file checking with requireConfigFile: false, or configure Babel so that it can find the config files

✖ 1 problem (1 error, 0 warnings)

It's telling you we need to configure babel for @babel/eslint-parser to work.

Lets set up a babel config file.

Create a file called .babelrc and populate it with:

{
  "presets": [
    ["@babel/preset-env"]
  ]
}

You can read about @babel/preset-env on the Babel website.

Now if you run npm run lint again you will hit the final error:

/path/to/code/Animal.js
  2:4  error  Parsing error: /path/to/code/Animal.js: Support for the experimental syntax 'classPrivateProperties' isn't currently enabled (2:5):

  1 | export class Animal {
> 2 |     #noise = '';
    |     ^
  3 | 
  4 |     constructor(noise) {
  5 |         this.#noise = noise;

Add @babel/plugin-proposal-class-properties (https://git.io/vb4SL) to the 'plugins' section of your Babel config to enable transformation.
If you want to leave it as-is, add @babel/plugin-syntax-class-properties (https://git.io/vb4yQ) to the 'plugins' section to enable parsing

✖ 1 problem (1 error, 0 warnings)

You could proceed to add plugins for each of the proposals as the instructions say, alternatively you can opt to say I want all shipped proposals.

To do this change your .babelrc over to:

{
    "presets": [
      ["@babel/preset-env",
      {
        "shippedProposals": true
      }]
    ]
  }

From the Babel docs: "set the shippedProposals option to true. This will enable polyfills and transforms for proposal which have already been shipped in browsers for a while."

If you are using Jest

If you are using Jest, it will automatically pick up .babelrc files, this might be problematic, as it will very helpfully start to try to transpile things like async/await, potentially leading you down even more rabbit holes. With really helpful messages like:

ReferenceError: regeneratorRuntime is not defined

By dumb luck, i've been through the pain of this message many times, and knew exactly what was wrong, Jest was trying to transform the perfectly valid code.

It's almost 2021, and this is a server app, I certainly do not want to transpile async/await especially not in unit tests!

One way to work around this is to use a non-standard name for your .babelrc file e.g. .babel-eslintrc. There may be better solutions, but I certainly don't want Jest unnecessarily transforming code.

In your .eslintrc file you can update babelOptions to use a custom configFile

"babelOptions": {
    "configFile": "./.babel-eslintrc"
 }

And there we go, Jest is now happy again because it's not using the Babel configuration.

Summary

All in all this was a lot harder than I thought it would be, my guess is that many people don't hit this issue because they happen to already have Babel configured. But in the case of backend dev, getting along happily, just trying to make use of a shipped JavaScript feature in a server, you can get dragged into the hellscape of frontend development tooling, and no one has fun there.

I hope this was a good read, if you feel like reading more of my work, please follow me on Twitter @griffadev, or get me a coffee if you feel like it ☕.

Using the Web Share API and meta tags for native sharing

George Griffiths — Mon, 12 Oct 2020 14:41:06 +0000

Supporting all of various places that content can be shared on a web site is tricky, all sites have their own mechanisms for doing it, and you can never support all of the places that a user may want to share content.

Thats where the Web Share API comes in, with it, sharing content on the Web is much more straight forward, using this API also gives your users the freedom to share content wherever they like, from chat apps and social media to a notes app.

At the end of the article I go into the all important html meta tags, to make sure that sites/apps can pick up nice previews for your content.

if(navigator.share) {
    navigator.share({
            text:"Some text to share",
            title: "Title of what you are sharing",
            url: "url to share"
        }).then(() => {
            // do something on success
        }).catch(() =>  {
            // handle errors
        });
    })
}

And your user will end up with a "share sheet", where they can decide where they want to share.

Browser support

At this point you're probably thinking yeah, this is great and all but the browser support is terrible, but I think you might be surprised, the browser support is good in the place it matters most: mobile.

Yeah that is a lot of red, but if you analyze the data, the support is decent in key places:

Safari 14 on IOS AND Safari on Mac OS.
Chrome for Android
Firefox for Android
Edge (Windows only)
Samsung Internet

We're at around 55% of global users, however if you filter to just mobile you're looking at 91.96% according to caniuse.com. Sure, the obvious missing pieces are cross platform support for Chrome and Firefox, as well as key OSs to capture non Western demographics, but you can always fallback to the older methods of sharing on browsers that don't support the API, and thats what i'll go into next.

For a live demo of the code, check out this post on my website, you'll see the share links. If you are on a browser that supports the web share api you will see an additional share button: griffa.dev

Web Share API as progressive enhancement

The rest of this article describes the approach that I took for implementing the Web Share API as a progressive enhancement to my blog. For browsers where the API is supported the option is offered, otherwise I added share buttons for common places where I might expect people to share e.g. Twitter.

For my blog i'm using 11ty, a really flexible static site generator which encourages you to build lightweight and fast websites.
I wrote about my journey with 11ty over here: How I got started with 11ty.

Doing things the old way

Supporting lots of different websites for share targets is quite time consuming, each does it in different ways.

I started out with a bit of html templating:

<nav class="sharing" aria-label="Social Sharing">
  <ul class="social-share">
    {%- for link in external.sharing %}
              <li>
                <a class="icon-share url-share" title="{{link.text}}" target="_blank" href="{{link.url(title, tags, page) | url}}" class="action external">
                    {% include link.icon %}
                </a>
            </li>
      {%- endfor %}
  </ul>
</nav>

I'm making use of a few things here in 11ty, Nunjucks templating and global data.

Global data files are either JSON or javascript, you can see from the code snippet that i have a sharing array of objects and each object has the following:

text
icon
url function: passing in the page title, tags for my blog post and the current 11ty page object.

Let's take a look at how this is implemented; I have the following file src/_data/external.js, this is where the code earlier gets external from as its variable name.

The file looks like this.

module.exports = {
  sharing: [
    {
      icon: "img/twitter.svg",
      text: "Share to Twitter",
      url(title, tags = [], page) {
        const twitterUrl = "https://twitter.com/intent/tweet?text=";
        const {text, url} = genericShare.data(title, tags, page);
        return `${twitterUrl}${encodeURIComponent(`${text} ${url}`)}`;
      }
    },
    {
      icon: "img/linkedin.svg",
      text: "Share to LinkedIn",
      url(title, tags = [], page) {
        return `https://www.linkedin.com/shareArticle?mini=true&url=${getUrl(
          page
        )}&title=${encodeURIComponent(title)}&source=griffadev`;
      }
    },
    {
        icon: "img/reddit.svg",
        text: "Share to Reddit",
        url(title, tags = [], page) {
          const baseUrl = "https://www.reddit.com/submit?";
          return `${baseUrl}uri=${getUrl(page)}&title=${encodeURIComponent(title)}`;
        }
    },
    {
        icon: "img/awful.svg",
        text: "Share to Hacker News",
        url(title, tags = [], page) {
          const baseUrl = "https://news.ycombinator.com/submitlink?";
          return `${baseUrl}u=${getUrl(page)}&t=${encodeURIComponent(title)}`;
        }
    }
  ]
}

Hopefully, you can see how the html maps over to the javascript. Each of the websites share the content in slightly different ways.
You might be noticing genericShare.data is missing, you'd be right.

Let's take a look at that function:


const siteMeta = require("./metadata.json");
const helpers = require("./helpers");
const getUrl = (page) => encodeURIComponent(`${siteMeta.url}${page.url}`);

const genericShare = {
  data(title, tags = [], page) {
    const url = `${siteMeta.url}${page.url}`;
    const text = `${title} ${
      // get my twitter handle
      siteMeta.author.twitter} ${tags
      // remove 11t built in tags from the tags for my bpost
      .filter(helpers.filterCollectionTags)
      // add a twitter style hashtag
      .map((tag) => `#${tag}`)
      // convert back to a string
      .join(" ")}`;
    return {
      text,
      title,
      url
    }
  }
}

In this function i'm getting the full url for my website siteMeta.url from a metadata.json and appending that to the current page.url.
Also in my metadata.json I have some more data about myself e.g. my twitter handle, when people share my posts I can get automatically tagged.

Finally, I added all of the tags from my front matter, that are on the blog post, over to the text for the share content too, I filter out tags that I don't want, for example tags for my 11ty collection names.

If you aren't sure what i mean by front matter it's the metadata at the top of my markdown files:

---
title: 'Using the Web Share API for simple native sharing'
tags:
  - JavaScript
  - 11ty
  - Webdev
---

For a full implementation reference, checkout my Github repo for my blog:

Adding in the Web Share API

Now we've implemented the basic behavior for everyone, we can progressively enhance the website for browsers that support the Web Share API.

Updating the HTML template:

<nav class="sharing" aria-label="Social Sharing">
  <ul class="social-share">
    {% set genericShareData = external.genericShare.data(title, tags, page) %}
    <li class="native-share" style="display: none;" hidden>
        <button data-title="{{genericShareData.title}}" data-url="{{genericShareData.url}}"
                data-text="{{genericShareData.text}}" aria-label="Native share">
            {% include 'img/share.svg' %}
        </button>
    </li>
    {%- for link in external.sharing %}
        <li>
            <a class="icon-share url-share" title="{{link.text}}" target="_blank" 
                href="{{link.url(title, tags, page) | url}}" class="action external">
                {% include link.icon %}
            </a>
        </li>
    {%- endfor %}
  </ul>
</nav>

The key thing that has changed from the original example is the addition of:

    {% set genericShareData = external.genericShare.data(title, tags, page) %}
    <li class="native-share" style="display: none;" hidden>
        <button data-title="{{genericShareData.title}}" data-url="{{genericShareData.url}}"
                data-text="{{genericShareData.text}}" aria-label="Native share">
            {% include 'img/share.svg' %}
        </button>
    </li>

You'll see that similar to the example above i'm passing title, url and text using data-* attributes,
and executing a function my global data object external.genericShare.data(title, tags, page).

By default this content is hidden with display:none, we're going to enable the content with a little bit of JavaScript.

 if(navigator.share) {
    const nativeShare = document.querySelector('.native-share');
    if (nativeShare) {
        // make the button visible
        nativeShare.style.display = 'flex';
        nativeShare.querySelector('button').addEventListener('click', (e) => {
            const button = e.currentTarget;
            navigator.share({
                // grab the data attributes from the html
                text:button.getAttribute('data-text'),
                title: button.getAttribute('data-title'),
                url: button.getAttribute('data-url')
            }).then(() => {
                // show some content to say it was shared, e.g. thank the user.
                nativeShare.classList.add('shared');
            }).catch(() =>  {
            })
        });
    }
}

First, i'm checking that we have access to navigator.share. If its available, the button is made visible, a click handler is added and on click the data-* attributes are read, finally navigator.share is called.

If you wanted to, you could hide your fallback links when navigator.share is available, I chose not to just yet, but might do down the road, when more desktop site get support for the Web Share API.

If you're wondering how to go about adding JavaScript into an 11ty project, there are many ways to go about it, I recently wrote about how to approach this for Web Components.

Meta tags

To round all of this work out you will want to make sure that you have all of the appropriate meta tags set on your website so that sites can put the correct images/description in share link previews.

For example:

<meta name="author" content="{{author or metadata.author.name}}">
<meta property="og:site_name" content="{{ siteTitle }}"/>
<meta property="og:title" content="{{ pageTitle }}"/>
<meta property="og:type" content="website"/>
<meta property="og:url" content="{{ currentUrl }}"/>

{% if socialImage %}
    <meta name="twitter:card" content="summary_large_image"/>
    <meta property="og:image" content="{{ socialImage }}"/>
    <meta name="twitter:image" content="{{ socialImage }}"/>
    <meta property="og:image:alt" content="Page image for {{ metadata.title }}"/>
    <meta name="twitter:image:alt" content="Page image for {{ metadata.title }}"/>
{% endif %}

{% if tagline %}
    <meta name="description" content="{{ tagline }}"/>
    <meta name="twitter:description" content="{{ tagline }}"/>
    <meta property="og:description" content="{{ tagline }}"/>
{% endif %}

You can see my full metadata over on Github.

Summary

For a live demo of the code, check out this post on my website, you'll see the share links: griffa.dev

Support for the Web Share API is growing and with it progressive enhancement is a really great approach to building modern websites, I wrote about using this approach for fast sites, with interactivity sprinkled in here.