Chapter 8: HTML part three

Cover image by cottonbro

Recap

Last time, we continued working with PostHTML and did the following:

• Learned how to use the picture tag in HTML5
• Created a new fragment using the picture tag and passed it a couple of arguments
• Wrote some code to rebuild pages in the dist directory when we update them in the src directory
• Put the code which calls PostHTML into its own module

We ended last time with some tasks still to do:

• Handling changes to fragments
• Getting the dist directory set up correctly for Our Hypothetical Second Developer
• Adding content inside of modules

At the end of the last chapter, we wrote a watch-html task which used the onchange package to call a script. That script (html-update.js) attempted to call PostHTML and pass it the input and output paths for our HTML file.

This worked well, if the file being updated represented a HTML page. But what if we edit a HTML fragment when the task is watching?

Watching fragments

We don't know how often a particular fragment appears in different HTML pages, so the brute force approach is to rebuild all HTML pages when any single fragment is updated. This isn't ideal for a couple of reasons:

1. We'll be rebuilding some pages which don't need to be rebuilt
2. This operation requires a lot of writing to the hard drive of individual files and will be both slow and won't scale well

This is unfortunate and I can't see an easy fix. If we could use browserSync to tell us which page was currently open in the browser, we could just update that page, but that would still leave all the other pages out of date.

This means we need to write a task which rebuilds all HTML pages. Hmm. This is reminding me a lot of what we did with Sharp.

Generating all HTML

Create a new file called generate-all-html.js inside the tools directory.

It should look like this:

import getFiles from "./get-files.js";
import callPostHTML from "./call-posthtml.js";

const allHTMLPaths = getFiles('./src/views');

allHTMLPaths.map((path) => {
  console.log(path);
  // callPostHTML(something, somethingElse);
});

This is pretty similar to compress-all-images.js, but instead of importing write-images.js, we're importing our new function callPostHTML. Let's see what that console logs when we point it at something.

In a terminal, cancel any currently running commands and type:

node tools/generate-all-html.js

You'll probably see something like this in the terminal:

./src/views/index.html
./src/views/about/index.html

This path is needs a couple of tweaks, to get it in the format which callPostHTML() expects. Create two new variables inside the map() function and change the console.log() so it shows them to us:

allHTMLPaths.map((path) => {
  const inputPath = path.replace('./', '');
  const outputPath = inputPath.replace('views/', '').replace('src/', 'dist/');
  console.log(inputPath, outputPath);
  // callPostHTML(something, somethingElse);
});

Run the file using node tools/generate-all-html.js and you should see something like this:

src/views/about/index.html dist/about/index.html
src/views/index.html dist/index.html

Perfect! Let's plug those variables into callPostHTML() and try it out. Update the inside of your map() function to read:

allHTMLPaths.map((path) => {
  const inputPath = path.replace('./', '');
  const outputPath = inputPath.replace('views/', '').replace('src/', 'dist/');
  callPostHTML(inputPath, outputPath);
});

Now delete the contents of your dist folder and run node tools/generate-all-html.js (remember to try the up arrow trick to save yourself some time). It won't generate out the images, javascript or CSS, but you should (eventually) see all your HTML files appear inside the dist directory.

Choosing how much HTML to generate

We have two different scripts which do similar tasks:

• html-update.js runs when the user makes a change to any HTML file inside either views or fragments
• generate-all-html.js can be manually called in order to generate all the HTML files inside views

We already worked out that if we edit a file inside the fragments directory, we need to generate all HTML view files from scratch. How about we treat generate-all-html.js as a module, rather than a script?

This means we can import it into html-update.js and trigger a build if we notice a change to a fragment.

Changing generate-all-html.js into a module means assigning it to a variable, then exporting that variable as the default. Like this:

import getFiles from "./get-files.js";
import callPostHTML from "./call-posthtml.js";

const generateAllHTML = () => {
  const allHTMLPaths = getFiles('./src/views');

  allHTMLPaths.map((path) => {
    const inputPath = path.replace('./', '');
    const outputPath = inputPath.replace('views/', '').replace('src/', 'dist/');
    callPostHTML(inputPath, outputPath);
  });
};

export default generateAllHTML;

Now if we try and run it directly, it won't do anything. But we can call it from a different file instead.

Back to fragments again

Let's add a new variable to html-update.js, just after where we instantiate editedDistPath:

...
if (triggerEvents.includes(fileEvent)) {

  console.log("HTML change detected");

  const { distPath, fileName, extName } = getDistPath(srcPath);
  const editedSrcPath = srcPath.replaceAll('\\', '/');
  const editedDistPath = distPath.replace('/views', '');
  // Do we need to rebuild all HTML files?
  const rebuildAll = editedSrcPath.indexOf('fragments') > -1;
}
...

The new line looks a bit weird, doesn't it? This stores the result of a JavaScript comparison into a variable. What this means is that the comparison will either be true or false. Rather than storing the comparison itself, the variable will instead store the boolean value of the comparison. The comparison itself looks for the string fragments inside the editedSrcPath variable. If this string is not present, the indexOf() function will return -1. If it's present then the function will returns the index within the string where the fragment appears. So by checking if it is greater than minus one, we can determine that the string appears somewhere inside editedSrcPath.

Using rebuildAll to decide what to do

Put an if/else statement around the call to callPostHTML(), like this:

if (rebuildAll) {
  // rebuild all HTML
} else {
  // Pass `callPostHTML()` our paths
   callPostHTML(editedSrcPath, `${editedDistPath}/${fileName}${extName}`);
}

Let's import our new generateAllHTML() function into this file to take care of that first part.

At the top of html-update.js, import the function like this:

import generateAllHTML from "./generate-all-html.js";

Now call it from inside the if statement like this:

if (rebuildAll) {
  generateAllHTML();
  console.log(`Fragment ${fileEvent === 'add' ? "added" : "changed"} successfully`);
} else {
  callPostHTML(editedSrcPath, `${editedDistPath}/${fileName}${extName}`);
  console.log(`HTML page ${fileEvent === 'add' ? "added" : "changed"} successfully`);
}

Testing it out

Let's test this out! Start the watch task with:

npm run watch-html

Open src/fragments/picture.html and add an extra line break at the end of the file, then hit save.

You should see Fragment changed successfully in the terminal and your dist directory should add in an about directory with a html file inside it.

Building all HTML files on first run

There's still another use-case we haven't handled yet: what happens if our hypothetical second developer installs the site for the first time? We need to generate all the HTML files for them at the same time.

We can't just run generate-all-html.js because that does nothing but export the function generateAllHTML() now. Let's account for this inside html-update.js instead.

How about if we call html-update.js from the command line with no arguments, it should generate all the HTML files? Just before the triggerEvents.includes(fileEvent) if/else statement, add this line:

// If this script is called with no arguments, rebuild all HTML files
if (typeof srcPath === "undefined" && typeof fileEvent === "undefined") {
  generateAllHTML();
}

Or if we want to use a slightly more modern way to express this:

// If this script is called with no arguments, rebuild all HTML files
typeof srcPath === "undefined" && typeof fileEvent === "undefined" && generateAllHTML();

(Remember that the JavaScript parser goes from left to right and keeps going until it finds a statement which equates as false. This means if the first two statements are both true, then the function generateAllHTML() will be called.)

Updating package.json

Edit your html-build command in package.json:

...
"scripts": {
  ...
  "html-build": "node tools/html-update.js",
  ...
},
...

This will call html-update.js but without arguments. Let's add this new command into our prepare command:

...
"scripts": {
  ...
  "prepare": "concurrently \"npm run sass-prod\" \"node tools/compress-all-images.js\" \"npx webpack --config webpack.prod.mjs\" \"npm run html-build\"",
  ...
},
...

npm run html-build
node tools/html-update.js

To test the rebuild command out, delete the contents of your dist directory, cancel the watch task in your terminal and run either npm run prepare or npm install from the terminal (both run the same command). This should regenerate all of the contents of the dist directory from scratch, including the HTML files, images, CSS and JavaScript.

Passing complicated markup to a HTML module

I want to return briefly to the first fragment we wrote - head.html to show you a trick. We currently call this module like this:

...
<head>
  <title>Hello Worm</title>
  <module href="src/fragments/head.html"></module>
</head>
...

This lets us specify a title tag which is custom to the page and also add in any additional CSS and JavaScript we want. But there's another way to do the same thing. Remove the old head tag from the HTML page entirely and change the rest of the markup so it looks like this:

...
<module href="src/fragments/head.html">
  <title>Hello Worm</title>
</module>
...

posthtml-modules lets us pass content rich with HTML tags between the opening and closing module tags. But this won't work until we tell PostHTML we want to pass data this way. So update posthtml.json to this:

{
  "plugins": {
      "posthtml-modules": {
          "root": "./src/views",
          "initial": true,
          "locals": true
      },
      "htmlnano": {}
  }
}

(the new bit is "locals": true)

Now change the head.html fragment so it looks like this:

<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="stylesheet" href="/css/main.css">
  <content></content>
</head>

Note that anything inside the module tag on individual HTML pages will be output where that <content></content> tag sits.

Now regenerate the HTML by running this from the terminal:

npm run html-build

The markup should be the same (although the order of the tags within the head is slightly different). This technique lets us simplify the tags on each page in the src/views directory while still giving us freedom to add tags within the head section of the page.

Nesting modules

Let's create a new fragment which contains a legal disclaimer. Create a file called disclaimer.html inside fragments. It should read as follows:

<p><small>The following site is intended for educational purposes only and should not be taken as legal advice. Your home may be at risk if you do not keep up repayments. Other websites are available. This website is presented "as is" and should not be printed on a t-shirt. Yea, though I walk through the valley of the shadow of death, I will fear no evil: for thou art with me; thy rod and thy staff they comfort me. The views presented on this site do not necessarily reflect those of Tim Berners-Lee, inventor of Alexander Graham Bell's Electric Internet.</small></p>

Now create a second fragment called footer.html. It should look like this:

<footer>
  <content></content>
</footer>

You can import these into the home page (src/views/index.html) just before the script tag at the bottom of the body like this:

<module href="src/fragments/footer.html">
  <module href="src/fragments/disclaimer.html"></module>
</module>

This might seem a little pointless - why not put this into one fragment? Well, at some point you might have more than one style of footer or you might want to re-use your legal disclaimer elsewhere on the site (such as on a sign up form).

Wait! That's not all we can do!

In src/views/index.html change your call to the module to this:

<module href="src/fragments/footer.html">
  <h2>Home page legal disclaimer</h2>
</module>

... then in src/fragments/footer.html, update it to this:

<footer>
  <content></content>
  <module href="./disclaimer.html"></module>
</footer>

Now we can pass custom markup to the footer when we call it, but the legal disclaimer can be the same on every page. Perhaps some pages require other kinds of legal disclaimers, I don't know. I've no idea who your site is for.

Note that the path to disclaimer.html is different in the above example because it's looking for it in the same directory as where the footer.html lives.

Look, not all of my examples can be good, OK?

Summary

• We started the chapter by working out how we should handle what happens when a developer edits a fragment
• What we need to do is very similar to what we did with images, so we reused some code from that
• We changed generate-all-html.js into a module and started treating it like a function, rather than a script to be run
• This means html-update.js can either update one file or all the files, depending on what arguments are used
• We changed the way fragments worked so we could call them, but add large chunks of markup inside
• Then we tried a couple of other ways of nesting fragments where content appears on more than one page

View Chapter 8 code snapshot on GitHub

Quiz

The end

Thanks for making it all the way to the end of this course. I don't care what anyone else says, I think you're a genius. I want to thank Ewan Burns for inspiring this course and helping me develop it.

Stretch goals

Auto-resizing images

For certain kinds of module - for example hero panels - we need images which are a specific width and height. What if you had different rules for different directories inside the src/img directory? For example, if a developer dropped a large image inside a directory called src/img/hero, image-compress.js could notice this and call a module which resized the image automatically, before generating out the different formats.

What if we took this further? Why force mobile users to download the desktop images? Expand out the picture tag so that it uses CSS media queries to show different images at different browser widths. That way, each hero panel might generate nine different images, which would only display at certain breakpoints.

Linting

What if we checked our code before we checked it in? Using so-called linting packages will warn us if we add obvious bugs or bad practice?

TypeScript

What if we wanted to write a version of JavaScript which is more robust and can show us errors as we type? TypeScript was originally developed to enforce data types more strictly in JavaScript but has since become somewhat of an industry standard for large web applications.