Forem: Carl Mungazi

Solving The Same Algorithmic Challenge In 13 Different Ways

Carl Mungazi — Wed, 15 Sep 2021 19:09:33 +0000

Given an array (the first argument in the destroyer function) remove all elements from the initial array that are of the same value as these arguments in the function. Note: You have to use the arguments object

Algorithmic challenges such as the one above (from FreeCodeCamp) are fun ways to stretch your problem solving muscles. You can usually come up with a few solutions but what if you tried solving it in as many ways as you can?

I have always wanted to do that so when I found myself with some free time, I began dreaming of ways I could torture JavaScript into doing things which would be a sackable offence if they ever found their way into a production codebase.

The challenge

The problem itself is fairly straightforward. You have a function which takes multiple inputs (the target array plus one or more values) and returns an array which contains the target array minus the values entered as the other arguments. From this explanation we can deduce the following:

The solution will involve array manipulation
The solution has to be able to handle any number of arguments (via the arguments object)

Solution 1: Make it work

function destroyerForEach(arr, ...args) {
  return arr.filter((el) => {
    let passed = el;
    args.forEach((num) => {
      if (num === el) passed = null;
    });
    return passed !== null;
  });
}

Whenever I am solving a problem, I create a working solution as fast as possible and then improve it afterwards. destroyerForEach takes the more long-winded approach of filtering through the target array and then looping through the rest of the arguments for each item in the target array. It is not pretty but it works. You could improve your programming street cred with this one liner args.forEach(num => num === el ? passed = null: null) in the .forEach function.

Solution 2: Filter and find

function shouldItemBeDestroyed(targetElement, comparisonArr) {
  return comparisonArr.find((el) => el === targetElement);
}

function destroyer(arr, ...args) {
  return arr.filter((el) => el !== shouldItemBeDestroyed(el, args));
}

If the .forEach and .filter combination is not to your taste, you can reach for .find. This solution has the added benefit of splitting the logic between different functions, thus improving the testability of the code. Again, you could unleash your inner one line maximalist with this:

const destroyer = (arr, ...args) =>
  arr.filter((el) => el !== args.find((item) => item === el));

Solution 3: Short and simple

function destroyerIncludes(arr, ...args) {
  return arr.filter((item) => !args.includes(item));
}

This solution gets to the crux of the matter without much ceremony. You will notice that .filter has been a mainstay in each solution thus far. This is because it is perfectly suited to tasks such as this. An interesting thing to note is that .includes returns a boolean whilst .filter's testing function returns a value which coerces to either true or false. This is useful if you like to avoid implicit coercions. You can take this solution to new heights by indulging both your ES6 and one liner tendencies to create this beauty:

const destroyerIncludes = (arr, ...args) =>
  arr.filter((item) => !args.includes(item));

Solution 4 & 5: Indexing

function destroyerIndexOf(arr, ...args) {
  return arr.filter((item) => args.indexOf(item) < 0);
}

// OR

function destroyerLastIndexOf(arr, ...args) {
  return arr.filter((item) => args.lastIndexOf(item) < 0);
}

We can continue keeping things simple by using array indexes to determine which values need purging. This solution only works if we use the spread syntax to turn the arguments object from an Array-like object into an array. We also need to perform this spread operation in the parameter declaration. Had we done it like this, for example:

function destroyerIndexOf(arr) {
  const args = [...arguments];
  // ... rest of the code goes here
}

destroyerIndexOf([1, 2, 3, 4], 2, 3);
// args would be [ [ 1, 2, 3, 4 ], 2, 3 ]

We would be including the target array in our array of elimination values.

Solution 6: Give the filter some

function shouldItemBeDestroyed(target, comparisonArr) {
  return comparisonArr.some((el) => el === target);
}

function destroyerSome(arr, ...args) {
  return arr.filter((el) => !shouldItemBeDestroyed(el, args));
}

A similar solution to the one using .find, with the difference being .some returns a boolean instead of a value.

Solution 7: #nofilter

function destroyerValuesIterator(arr, ...args) {
  let finalArr = [];
  const iterator = arr.values();

  for (const value of iterator) {
    if (!args.includes(value)) finalArr.push(value);
  }

  return finalArr;
}

What's that? No filtering?! Yes, it is possible to live without .filter and we do that by relying on for..of to handle the iteration. The .values methods returns an Array Iterator object which contains the values for each index in the array.

Solution 8: ?!?!?!

function destroyerArrOwnProp(arr, ...args) {
  args.forEach((item) => {
    Object.defineProperties(Array, {
      [item]: {
        value: item,
        writable: true,
        configurable: true, // so we can use delete to clean up after ourselves
      },
    });
  });

  return arr.filter((item) => {
    return !Array.hasOwnProperty(item);
  });

I cannot think of a scenario where this is even an option but it is reassuring to know we can create such monstrosities should the mood strike. Here we are extending the built-in Array object so we can use the .hasOwnProperty method later to weed out the repeated values. In this solution's defence, it sets the configurable property to true so we can cover our tracks by deleting the properties and pretending this never happened.

Solution 9: Splicin' it up

function destroyerSpliceAndFromAndForEach(arr, ...args) {
  const copiedArr = Array.from(arr);

  arr.forEach((item) => {
    args.forEach((num) => {
      if (num === item) {
        const index = copiedArr.indexOf(item);
        copiedArr.splice(index, 1);
      }
    });
  });

  return copiedArr;
}

Here we use Array.from to create a shallow copy of the target array and then enlist the services of .splice to hack away the repeat elements. We can safely perform surgery on copiedArr because whilst it has the same values as arr, they are different array objects, so we don't have to worry about any mutation.

Solution 10: Functional preparation

function destroyerFromMap(arr, ...args) {
  const mapFn = (item) => ({ value: item, isSameVal: args.includes(item) });
  const copiedArr = Array.from(arr, mapFn);

  return copiedArr.filter((item) => !item.isSameVal).map((item) => item.value);
}

We're not done with .from just yet. This method has two optional arguments, the first of which is a map function that is called on every element of the array being copied. We can take advantage of this to prepare our array during the copy process by creating an object and adding a property on it which checks if the item being filtered against the arguments.

Solution 11: Let's get reducin'

function destroyerReducerConcat(arr, ...args) {
  return arr.reduce((seedArray, elementFromSourceArr) => {
    if (!args.includes(elementFromSourceArr)) {
      return seedArray.concat(elementFromSourceArr);
    }

    return seedArray;
  }, []);
}

This is one of my favourite solutions because it taught me a new way of using the .reduce method. For a more in-depth and comprehensive explanation of the mechanics behind this solution, this article has you covered. With the .reduce method, we can either provide a second argument or omit it, in which case it defaults to the first element of the array being worked on. In our case, we can "seed it" with an empty array and then populate that array with the passing elements. The other new method which makes its first appearance is .concat and we use it to merge values into the seed array.

Solution 12: Let's get fancy with our reducin'

const destroyerReducerOneLinerSpread = (arr, ...args) =>
  arr.reduce(
    (seedArray, elementFromSourceArr) =>
      !args.includes(elementFromSourceArr)
        ? (seedArray = [...seedArray, elementFromSourceArr])
        : seedArray,
    []
  );

As if solution 11 wasn't fancy enough, we can really flex our ES6 muscles by swapping .concat for the spread operator and using a ternary to really drive home that we can write one-liners with the best of them.

const destroyerReducerOneLinerSpread = (arr, ...args) =>
  arr.reduce(
    (seedArray, elementFromSourceArr) =>
      !args.includes(elementFromSourceArr)
        ? (seedArray = [...seedArray, elementFromSourceArr])
        : seedArray,
    []
  );

Solution 13: Settin' things up

function destroyerSet(arr, ...args) {
  const argsSet = new Set(args);
  let uniqueVals = [];

  for (let i = 0; i < arr.length; i++) {
    if (!argsSet.has(arr[i])) uniqueVals = [...uniqueVals, arr[i]];
  }

  return uniqueVals;
}

The final solution is another gratuitous use of a random JavaScript feature. Here we have cajoled a Set into storing our arguments and then used a for loop to iterate through the Set and find the unique values.

How does the ES6 Job Queue Work?

Carl Mungazi — Wed, 14 Oct 2020 14:24:05 +0000

When Promises were first introduced in ES6, they made the job of writing asynchronous code easier. Callback hell was replaced with simpler constructs that allowed developers to more easily handle asynchronous tasks. The key to understanding promises is knowing how the job queue (also known as the microtasks queue) works in JavaScript.

We will start by looking at some code:



function firstFunction() {
  thirdFunction()

  const firstResponse = Promise.resolve('1st Promise');
  const secondResponse = Promise.resolve('2nd Promise');

  setTimeout(() => {
    firstResponse.then(res=> {
      console.log(res);
    })
  })

  secondResponse.then(res=> {
    console.log(res);
  })
}

function thirdFunction() {
  const thirdResponse = Promise.resolve('3rd Promise');
  const fourthResponse = Promise.resolve('4th Promise');

  queueMicrotask(() => {
    console.log('Hello from the microtask queue')
  })

  thirdResponse.then(res=> {
    console.log(res);
  })

  setTimeout(() => {
    fourthResponse.then(res=> {
      console.log(res);
    })
  })
}

function secondFunction() {
  let i = 0;
  let start = Date.now();

  for (let j = 0; j < 5.e9; j++) {
    i++;
  }
  console.log("Loop done in " + (Date.now() - start) + 'ms');
}

setTimeout(() => {
  console.log('first timeout')
});

firstFunction()
secondFunction()
console.log('first console log')

In what order should we expect the logs to appear?

Enter the event loop

It maybe surprising to learn that the ECMAScript specification does not mention the event loop. Instead, the event loop refers to the way in which code is processed by a browser's JavaScript engine. JavaScript runs on a single-threaded model so only one task can be processed at any moment in time. This obviously can lead to complications. What happens if a mouseover event is triggered just before a timer started by setTimeout expires? Or if you fire a network request and the response comes in the middle of the browser re-rendering the UI?

The diagram below shows the different parts of the browser that work together to manage this asyncronicity.

The event loop performs its work in iterations or "ticks". JavaScript code is executed in a run-to-completion manner (the current task is always finished before the next task is executed), so each time a task finishes, the event loop checks if it is returning control to other code. If it is not, it runs all of the tasks in the job queue and then runs the tasks in the task queue. We can better illustrate this by applying it to our example code.



// ... firstFunction, secondFunction and thirdFunction declarations have been omitted for brevity

setTimeout(() => {
  console.log('first timeout')
});

firstFunction()
secondFunction()
console.log('first console log')

When firstFunction is executed, the browser's internal state is:

If setTimeout is called without a specified duration, it defaults to 0 milliseconds. setTimeout itself is a browser API so it does not appear in the call stack but the callback it returns is placed into the task queue ready to be called during a future event loop iteration.

The first thing firstFunction does is call thirdFunction, which looks like:



function thirdFunction() {
  const thirdResponse = Promise.resolve('3rd Promise');
  const fourthResponse = Promise.resolve('4th Promise');

  queueMicrotask(() => {
    console.log('Hello from the microtask queue')
  })

  thirdResponse.then(res=> {
    console.log(res);
  })

  setTimeout(() => {
    fourthResponse.then(res=> {
      console.log(res);
    })
  })
}

This is where things get interesting. In the code above we resolve two promises and assign their resolved values. Using the then method on each promise, we specify the function which should run once it settles. A settled promise is a promise which has moved from pending (when it is performing an underlying process such as fetching data) to either fulfilled (success) or rejected (error). When it settles, it queues a microtask for its callback. queueMicrotask is self-explanatory. It is a more direct way of queuing a microtask.

Once thirdFunction has finished executing, control is handed back to firstFunction which also finishes running its code. After this, the browser's internal state is:

It is worth noting that our program has not done anything yet at this point despite running two functions. When working with asynchronous code these are the nuances which can confuse developers, especially when you consider that the next line our code, secondFunction, mimics the behaviour of blocking code by running a loop which last for a few seconds. Despite having six callbacks queued, the console.log statement in secondFunction is the first thing printed to the console and it is followed by the last line of the script, which is another log statement.

At this stage, the event loop reaches the end of its current iteration, so it looks to the job queue and runs the callbacks there in a first-in-first-out manner. It is possible for the code in the job queue to schedule more callbacks. However, these will not be deferred until future iterations but will instead run in the current iteration, meaning it is possible to starve your program by creating an endless loop of job queue callbacks. At the end of the first iteration, the following will have been logged to the console:



Loop done in 5672ms
first console log
Hello from the microtask queue
3rd Promise
2nd Promise

And the browser's internal state is:

You will notice that the callbacks for both the first and fourth promises were never put into the job queue. This is because instead of calling then on them directly, we put them in the callback for a setTimeout function. So when the event loop begins its second iteration, it will first look at the the task queue. The setTimeout callbacks for our promises will each queue another callback for the job queue which will be run at the end of current iteration. This means when our program has finished running, the following is the order in which things are logged to the console. If any of the code in the second iteration queued more stuff for the task queue, it would not be run until a future iteration.



Loop done in 5672ms

first console log

Hello from the microtask queue

3rd Promise

2nd Promise

first timeout

4th Promise

1st Promise

Summary

Whilst the explanation above is not an exhaustive look at the job queue (for example, we never covered how it is used by the MutationObserver API) but I hope it has improved your understanding of the job queue in relation to promises. When writing code it is easy to forget that its execution order is not always straightforward, especially when it runs in a browser environment where other tasks outside your control could happen.

Learn JavaScript by building a UI framework: Part 6 - Intro to Virtual DOM Algorithms

Carl Mungazi — Wed, 10 Jun 2020 04:17:31 +0000

This article is the 6th in a series of deep dives into JavaScript. You can view previous articles by visiting the Github repository associated with this project.

This series does not comprehensively cover every JavaScript feature. Instead, features are covered as they crop up in solutions to various problems. Also, every post is based on tutorials and open source libraries produced by other developers, so like you, I too am also learning new things with each article.

In the last article we extended Aprender's functionality by adding events to DOM elements. In this post, we will take the first step in tackling the meatiest part of a UI framework - dynamically changing DOM elements.

As always, we will begin with some example code. Given the two objects below which represent a UI, how do we change from the old to the new?

{ // old dom
  type: "div",
  attrs: {},
  children: [
    "Search",
    {
      type: "p",
      attrs: {},
      children: []
    }
  ]
}

{ // new dom
  type: "div",
  attrs: {},
  children: [
    "No Search",
    {
      type: "span",
      attrs: {},
      children: []
    }
  ]
}

At the heart of any UI framework (or secret sauce, if you like) is the way changes are detected and made. In a typical application, the underlying framework devotes a lot of time to figuring out what changed, how it changed and how the change needs to be applied. This issue on the React repository, for instance, is a detailed technical discussion about the matter. There are many virtual DOM implementations out there and to construct our own we will look to simple-virtual-dom for inspiration.

So...what changed?

The "what changed?" part of the implementation begins with the following function:

function diff(oldTree, newTree) {
  const patches = {};
  const index = 0;

  performDiff(oldTree, newTree, patches, index)

  return patches;
}

The arguments oldTree and newTree are object representations of the UI - the old state and the new state. In our case, we are changing the text from "Search" to "No Search" and the paragraph element to a span element. The patches object stores the differences between both states and it will be populated when we find the changes using performDiff. We will be making recursive calls as part of the diffing process so index acts as a counter to keep track of the current diff iteration. Finally, we return the patches object. It will be used later to make the changes. Let us look at performDiff:

function performDiff(oldTree, newTree, patches, index) {
  const currentPatch = [];

  if (newTree === undefined) {
    // we do nothing here because the final else statement will deal with it
  } else if (typeof oldTree === 'string' && typeof newTree === 'string') {  
    if (oldTree !== newTree) {
      currentPatch.push({
        type: 'TEXT',
        content: newTree
      })
    }
  } else if (oldTree.type === newTree.type) {
    diffChildren(oldTree.children, newTree.children, patches, index)
  } else {
    currentPatch.push({
      type: 'REPLACE',
      node: newTree
    })
  }

  if (currentPatch.length) {
    patches[index] = currentPatch
  }
}

performDiff is called recursively when we diff through any children, so currentPatch holds the changes belonging to the current iteration. The bulk of this function's work is done by a series of if statements which stem from the following questions:

Do we have a new DOM tree/element to diff against?

If we do not, we do nothing because the else clause of our if statement will handle that.

Are we comparing text nodes?

If we are dealing with text nodes, we will only make changes if the text is different. This change is recorded via an object that holds information about the type of change and the content related to that change.

Do we need to compare the children?

This is where the recursive fun begins. The diffChildren function is:

function diffChildren(oldChildren, newChildren, patches, index) {
  oldChildren.forEach((oldChild, idx) => {
    index++
    performDiff(oldChild, newChildren[idx], patches, index)
  })
}

It is very basic. Other frameworks shine here because they have to consider a host of issues. For instance, if none of the children have changed, is there any way of skipping this call? What is the most efficient way of comparing the child elements?

Are we comparing different elements?

If we are diffing two completely different DOM elements, the simplest thing to do is replace the old one with the new.

The algorithms of most UI frameworks can be boiled down to a similar list of questions.

In our case, the example code will go down the child comparison route, so let us walk through that:

First, diffChildren receives an array of children from the elements we are comparing. For each child, it recursively calls performDiff. However, before doing so it increments the index counter. In our example it goes from 0 to 1.

We call performDiff with the arguments "Search" and "No Search" as our comparison elements. As we are comparing text nodes, we create this object:

{
  type: "TEXT",
  content: "No Search"
}

and store it in the currentPatch array that is initialised whenever performDiff is invoked. This array keeps track of any changes that need to be made and if there are, at the end of the function they are assigned to the patches object with index being used as the key.

We repeat this process for the second child and once performDiff has completed its work, it leaves us with the following object:

{
  1: [
    {
      type: "TEXT",
      content: "No Search"
    }
  ],
  2: [
    {
      type: "REPLACE",
      node: {
        type: "span",
        attrs: {},
        children: []
      }
    }
  ]
}

This object, returned by the diff function, represents the changes we want to make to our UI. You can think of this as the first phase of our update process. In the second phase we will apply these changes to the DOM. This two-step process is similar to how React does things.

We will begin applying our changes with these two functions:

function patch(rootDomNode, patches) {
  const index = 0;
  performPatches(rootDomNode, patches, index)
}

function performPatches(node, patches, index) {
  const currentPatches = patches[index];

  if (node.childNodes) {
    node.childNodes.forEach(node => {
      index++
      performPatches(node, patches, index)
    });
  }

  if (currentPatches) {
    applyPatches(node, currentPatches)
  }
}

The workflow here should be familiar. patch takes the DOM element which is being updated and then calls performPatches with the changes and a counter. In performPatches we first perform any changes on child elements before making changes on the target element.

function applyPatches(node, currentPatches) {
  currentPatches.forEach(patch => {
    switch (patch.type) {
      case 'TEXT': {
        if (node.textContent) {
          node.textContent = patch.content
        }
        break;
      }
      case 'REPLACE': {
        const newNode = render(patch.node);
        node.parentNode.replaceChild(newNode, node);
        break;
      }
    }
  })
}

The humble switch case statement is at the core of the second phase of our update process. When replacing elements, we call upon Aprender's render function to create the DOM element for us.

And that is it! With the diff and patch functions we can now update DOM elements. If we were to write this as a proper application, it would be something like:

const aprender = require('aprender');

const oldTree = aprender.createElement('div', { 
    children: ['Search', aprender.createElement('p')] 
  }
);

const newTree = aprender.createElement('div', { 
  children: ['No Search', aprender.createElement('span')] 
  }
);

const root = aprender.render(oldTree)
aprender.mount(root, document.getElementById('app'))

const diff = aprender.diff(oldTree, newTree);

setTimeout(() => {
  aprender.patch(root, diff);
}, 5000)

Summary

Our algorithm is nothing more than a series of conditional checks. If you look at the heart of an UI framwork's update mechanism, you will find the same. However, our implementation is still missing a fundamental piece - the ability to perform updates automatically and in response to data changes. We have to use setTimeout to trigger the change detection process. We will fix that in the next post.

A glimpse into the mind of a framework author

Carl Mungazi — Fri, 29 Nov 2019 17:26:30 +0000

This article was first published on Git Connected.

Have you ever read the source code of a popular library and come across comments which explain what the code is doing? Sometimes the explanation given is straightforward but other times it leaves you scratching your head. I had one such moment recently and after much head scratching, it left me with a deeper appreciation of the effort that goes into open source tools and inspired me to write this post.

I came across the code in question whilst investigating how frontend frameworks attach event handlers to DOM elements. Over the last few months, I have been rebuilding different parts of the frontend stack as a way of improving my knowledge, and part of that has involved creating a UI framework based on the virtual DOM paradigm.

The code I was looking at was from Mithril. Internally, it registers events on DOM elements by creating an object and passing it as the second argument to the document.addEventListener method. The relevant source code is:

// Here's an explanation of how this works:
// 1. The event names are always (by design) prefixed by `on`.
// 2. The EventListener interface accepts either a function or an object // with a `handleEvent` method.
// 3. The object does not inherit from `Object.prototype`, to avoid
//    any potential interference with that (e.g. setters).
// 4. The event name is remapped to the handler before calling it.
// 5. In function-based event handlers, `ev.target === this`. We replicate
//    that below.
// 6. In function-based event handlers, `return false` prevents the default
//    action and stops event propagation. We replicate that below.
function EventDict() {
  // Save this, so the current redraw is correctly tracked.
  this._ = currentRedraw
}
EventDict.prototype = Object.create(null)
EventDict.prototype.handleEvent = function (ev) {
  var handler = this["on" + ev.type]
  var result
  if (typeof handler === "function") result = handler.call(ev.currentTarget, ev)
  else if (typeof handler.handleEvent === "function") handler.handleEvent(ev)
  if (this._ && ev.redraw !== false) (0, this._)()
  if (result === false) {
    ev.preventDefault()
    ev.stopPropagation()
  }
}

The third point: “The object does not inherit from Object.prototype, to avoid any potential interference with that (e.g. setters).” is what caught my attention. I reached out to Mithril core maintainer Isiah Meadows via the Mithril Gitter chat and his explanation is what we will go into next.

Guarding against rogue third-party code

Imagine that instead of inheriting from null, EventDict's prototype is Object.prototype. A Mithril user then writes the following code or uses a library whch does the following:

Object.defineProperty(Object.prototype, "onsubmit", {
  get() {
    return (e) => { 
      e.preventDefault();
      console.log('I am on the setter');
    }
  },
  set(val) {
    console.log(val);
  }
})

The application is then written as follows:

m.render(document.getElementById('app'), 
  m('form', {
    onsubmit: (e) => { 
      e.preventDefault(); 
      console.log('I am on the form element')
    }
  }, [
    m('button', { type: 'submit'}, 'Submit')
  ])
)

If the form is submitted, I am on the setter will be logged to the console instead of I am on the form. The contents of the getter and setter functions are not important but their existence means the onsubmit event handler is not registered on the form as intended.

This is because when the event is triggered, the EventDict.prototype.handleEvent method is executed and this line var handler = this["on" + ev.type] returns the onsubmit function on Object.prototype instead of the function specified on the form element.

The problem above also crops up when Mithril runs in an environment where Object.prototype has been extended in a much simpler fashion like so:

Object.prototype.onsubmit = 1

And the application code is:

m.render(document.getElementById('app'), 
  m('form', {
    onreset: (e) => { 
      console.log('resetting form...')
    },
    onsubmit: (e) => { 
      e.preventDefault(); 
      console.log('submitting form...')
    },
  }, [
    m('button', { type: 'reset'}, 'Reset'),
    m('button', { type: 'submit'}, 'Submit')
  ])
)

In this example, we have added the onreset event to our form. For us to understand the problems it poses, we first have to look at Mithril's updateEvent method. This method runs whenever on-event handlers are being set or removed on DOM elements.

function updateEvent(vnode, key, value) {
  if (vnode.events != null) {
    if (vnode.events[key] === value) return
    if (value != null && (typeof value === "function" || typeof value === "object")) {
      if (vnode.events[key] == null) vnode.dom.addEventListener(key.slice(2), vnode.events, false)
      vnode.events[key] = value
    } else {
      if (vnode.events[key] != null) vnode.dom.removeEventListener(key.slice(2), vnode.events, false) 
      vnode.events[key] = undefined
    }
  } else if (value != null && (typeof value === "function" || typeof value === "object")) {
    vnode.events = new EventDict()
    vnode.dom.addEventListener(key.slice(2), vnode.events, false)
    vnode.events[key] = value
  }
}

Virtual DOM frameworks like Mithril turn calls such as m('button', { type: 'reset'}, 'Reset') into objects which represent a given DOM element. In Mithril, these objects are called vnodes (For comparison, in React those objects are called fibers. I have written about them here). The key is the event name and the value is whatever object or function assigned to handle that event.

When the code is executed, the vnode object for the form element is passed to the updateEvent function. Inside the function, the else if clause runs because the vnode's events property is undefined. Once the function has finished executing, the vnode object will look like this:

{
  attrs: {
    onreset: onreset(e) { /* ... */ }, 
    onsubmit: onsubmit(e) { /* ... */ },
  }
  children: [{…}, {…}]
  dom: form
  domSize: undefined
  events: EventDict {
    onreset: onreset(e) { /* ... */ },
    _: undefined
  }
  instance: undefined
  key: undefined
  state: {}
  tag: "form"
  text: undefined
}

The events property has been assigned an EventDict instance and that instance is then given a reference to the onreset function as one of its properties. Also, the form DOM element has an event listener attached for the onreset event. So far, so good.

updateEvent runs for every on-event present on a DOM element. The second time it is called, it is passed onsubmit as the key argument. This time, however, the first if clause runs because vnode.events is no longer null.

function updateEvent(vnode, key, value) {
  if (vnode.events != null) {
    if (vnode.events[key] === value) return
    if (value != null && (typeof value === "function" || typeof value === "object")) {
      if (vnode.events[key] == null) vnode.dom.addEventListener(key.slice(2), vnode.events, false)
      vnode.events[key] = value
    }
    // ...
  }
  // ...
}

Within this if statement are two other if statements. The first one is skipped because vnode.events[key] is not equal to value, which in this case is the onsubmit function. In fact, the value of vnode.events[key] is 1. Why? Remember that this code is running in an environment where somebody has written this: Object.prototype.onsubmit = 1. Since the form EventDict instance does not yet have an onsubmit property and it inherits from Object.prototype, the JavaScript engine will walk up the prototype chain and find that onsubmit exists on Object.prototype.

The next if clause then checks that value has a function or object. This check passes but crucially, an event listener for the onsubmit event is not then attached to the form element because vnode.events[key] is not null as it should be, its value is 1. The next line adds the onsubmit property to the EventDict instance on the form element vnode along with the related function.

Like the first example where Object.defineProperty was used to add the onsubmit property to Object.prototype, when the form is submitted it will not behave as the developer intended.

Summary

Both the examples above seem like bizarre things to guard against but that is the kind of defensive coding framework and library authors have to do. Using the constructor function approach to create a new object which does not inherit from null means the newly created object is at the mercy of whatever additions may have been made to Object.prototype.

Learn JavaScript by building a UI framework: Part 5 - Adding Events To Dom Elements

Carl Mungazi — Fri, 11 Oct 2019 16:59:41 +0000

This article is the fifth in a series of deep dives into JavaScript. You can view previous articles by visiting the Github repository associated with this project.

At this stage in our project we have built a basic UI framework (Aprender), test library (Examinar) and module bundler (Maleta). We have not touched our framework for a while so in this post we will return to it. The most exciting thing Aprender can do is create and render DOM elements, so what more can we make it do?

Every development tool is built to solve a particular problem and our framework is no different. Its primary purpose is to be an educational tool but for that education to be effective, it needs to happen in the context of something. That something will be a search application that allows users to choose from a selection of these free public APIs, search for something and then display the results. We will incrementally build functionality which handles this specific use case instead of worrying about our framework meeting the large number of requirements for a production level tool. For example, production standard UI libraries have to handle all the various quirks and requirements of every DOM element. Aprender will only handle the elements needed to create the application.

The first order of business is to list the user stories for our search app:

As a user, I can view the search app
As a user, I can select an API
As a user, after selecting an API, I can view information explaining the API and what search parameters I can use
As a user, I can type in the search field and click the search button
As a user, after clicking the search button I can view the search results
As a user, I can clear the search results

We will also refactor our demo app to reflect the new goal:

const aprender = require('../src/aprender');

const Button = aprender.createElement('button', { 
    attrs: {
      type: 'submit'
    },
    children: ['Search'] 
  }
);
const Search = aprender.createElement('input', { attrs: { type: 'search' }});

const Form = aprender.createElement('form', {
    attrs: { 
      id: 'form',
      onsubmit: (e) => { 
        e.preventDefault(); 
        console.log('I am being submitted..') 
      }
    },
    children: [
      Search,
      Button
    ]
  },
);

const App = aprender.render(Form);

aprender.mount(App, document.getElementById('app'));

The only new addition in the code above is the function assigned to the onsubmit property of the form's attrs object and it is this functionality we will explore next.

Events and DOM elements

Adding event handling functionality to DOM elements is straightforward. You grab a reference to an element using a method such as getElementById() and then use the addEventListener method to set up the function that is called whenever an event is triggered.

For Aprender's event handling functionality, we will take inspiration from Mithril. In our framework, the renderElement function is responsible for attaching attributes to DOM elements, so we will put the event code there:

const EventDictionary = {
  handleEvent (evt) {
    const eventHandler = this[`on${evt.type}`];
    const result = eventHandler.call(evt.currentTarget, evt);

    if (result === false) {
      evt.preventDefault();
      evt.stopPropagation();
    } 
  }
}

function renderElement({ type, attrs, children }) {
  const $el = document.createElement(type);

  for (const [attribute, value] of Object.entries(attrs)) {
    if (attribute[0] === 'o' && attribute[1] === 'n') {
      const events = Object.create(EventDictionary);
      $el.addEventListener(attribute.slice(2), events)
      events[attribute] = value;
    }

    $el.setAttribute(attribute, value);
  }
  for (const child of children) {
    $el.appendChild(render(child));
  }

  return $el;
};

We are only interested in registering on-event handlers. Mithril and Preact both screen for these event types by checking if the first two letters of the attribute name begin with 'o' and 'n' respectively. We will do the same. addEventListener takes the event name as its first argument and either a function or object as the second argument. Typically, it is written like this:

aDomElement.addEventListener('click,' () => console.log('do something'));

Like Mithril, we will use an object but its creation will be different. Mithril's source has some comments which explain their approach and offer great insight into the considerations framework authors make when building their tools.

First, the event object is created using the new EventDict() constructor pattern as opposed to our Object.create(EventDictionary) approach. In Mithril, the object created whenever new EventDict() is called is prevented from inheriting from Object.prototype by this line:

EventDict.prototype = Object.create(null);

Mithril maintainer Isiah Meadows said one of the reasons this was done was to guard against third parties adding properties such as onsubmit or onclick to Object.prototype.

We are not worried about this so we create an object called EventDictionary which implements the EventListener interface. We then use Object.create to specify EventDictionary as the prototype and create an object which will hold a list of on-event handlers for the DOM element in question. Finally, the newly created object is assigned the attribute value.

After this, whenever an event is triggered on the DOM element in question, the handleEvent function on EventDictionary will be called and given the event object. If the event exists on the event object, it is invoked using call and we specify the DOM element as the this context and pass the event object as the only argument. If our handler's return value is false, the result === false clause will stop the browser's default behaviour and also prevent the event from propagating.

There is an excellent in-depth post which explains the differences of the Object.create approach over new Func() when creating objects. This Stack Overflow question also has some interesting thoughts on the two patterns.

A little bit about events

If we run our application, we should see an input field with a button next to it. Typing some text and clicking the button should log I am being submitted.. in our console. But if we remember, the first line in our form's onsubmit function is:

const Form = aprender.createElement('form', {
    // ...
      onsubmit: (e) => { 
        e.preventDefault(); 
        console.log('I am being submitted..') 
      }
    // ...
  },
);

What is e.preventDefault() and what does it do? The default behaviour when a form's onsubmit handler is called is for its data to be sent to the server and the page to be refreshed. Obviously, this is not always ideal. For one, you might want to validate the data before its sent or you might want to send the data via another method. The preventDefault function is a method on the Event object and it tells the browser to prevent the default action. However, if you were to programmatically create a form like this:

const form = document.createElement('form');
form.action = 'https://google.com/search';
form.method = 'GET';

form.innerHTML = '<input name="q" value="JavaScript">';

document.body.append(form);

Submitting the form by calling form.submit() would not generate the submit event and the data would be sent.

The next event we will look at is on our input field. We need to capture the input value so we can use it to make a request to the selected API. We have a few events we can choose from for this: oninput, onblur and onchange.

The onblur event fires when a focused element loses focus. In our case, it would fire only when the user focused away from the input field. The onchange event fires when the user changes the value of a form control, like our input field, and then focuses away from it. Finally, oninput is fired each time the value changes. This means every keystroke would fire the event. We will use the oninput event because it best suits our purposes. onchange and likewise onblur would be useful if we wanted to validate the input each time the search element lost focus. Note: if you were like me and did not know much about events when you first started using React, you would have been surprised to know that React's onchange event behaves exactly like oninput. There is even an issue about it.

Our final act will be to create a select element for our list of API options and attach an onchange event handler to it. And with that, our application code should look like this:

const aprender = require('../src/aprender');

const Button = aprender.createElement('button', { 
    attrs: {
      type: 'submit'
    },
    children: ['Search'] 
  }
);

const Search = aprender.createElement('input', { 
  attrs: { 
    type: 'search',
    oninput: (e) => console.log(e.target.value)
  }
});

const Form = aprender.createElement('form', {
    attrs: { 
      id: 'form',
      onsubmit: (e) => { 
        e.preventDefault(); 
        console.log('I am being submitted..')  
      }
    },
    children: [
      Search,
      Button
    ]
  },
);

const Dropdown = aprender.createElement('select', {
  attrs: {
    onchange: (e) => console.log(e.target.value)
  },
  children: [
    aprender.createElement('option', {
      children: ['--Please select an API--']
    }),
    aprender.createElement('option', {
      children: ['API 1']
    }),
    aprender.createElement('option', {
      children: ['API 2']
    })
  ]
});

const SelectAPI = aprender.createElement('div', {
  children: [
    aprender.createElement('h2', { children: ['Select API: ']}),
    Dropdown
  ]
})

const Container = aprender.createElement('div', {
  children: [
    SelectAPI,
    Form
  ]
})

const App = aprender.render(Container);

aprender.mount(App, document.getElementById('app'));

Summary

We have completed our first user story:

As a user, I can view the search app

In the next post we will tackle:

As a user, I can select an API.

This feature will expose us to the core reason why UI frameworks exist - keeping the user interface in sync with the application state.

How does your company assess bootcamp graduates?

Carl Mungazi — Fri, 20 Sep 2019 16:46:45 +0000

I am keen to hear how others have fared in assessing the abilities and potential of graduates from bootcamps such as Flatiron school, General Assembly etc.

Also, if you have graduated from a bootcamp, what do you think is a good way of assessing your technical knowledge and soft skills?

Do you enjoy reading source code?

Carl Mungazi — Fri, 20 Sep 2019 16:42:51 +0000

I love reading the source code of popular libraries and finding out how other developers have solved different problems. Just recently I was looking into how frontend frameworks attach event handlers to DOM elements and it led me to reading about the optimisations JavaScript engines make when accessing object properties.

Does anyone else loving digging around in other people's code?

Learn JavaScript by building a UI framework: Part 4 - Creating A Module Bundler

Carl Mungazi — Thu, 19 Sep 2019 17:29:20 +0000

This article is the fourth in a series of deep dives into JavaScript. You can view previous articles by visiting the Github repository associated with this project.

In the last article we added the functionality for our framework to create and render DOM elements, created an example application and then successfully tested it. Today we will cover the steps necessary to run our app in a browser.

The first step

At the moment, if we installed a server like http-server and spun it up in the folder housing our example application, this error shows up in the browser console Uncaught ReferenceError: require is not defined. This is because the require function only exists in the node environment. It provides a way of accessing code that exists in separate files. The easiest (and most painful) way to replicate this behaviour in the browser would be to use <script> tags.

Before the advent of ES Modules, developers used (and still do) either the CommonJS or AMD formats to tackle this problem. And this is where build tools such as Webpack or Parcel come in. Conceptually, their work is straightforward. They gather all the files needed to run an application, work out the dependencies of those files and then create one big JavaScript file which can run in a web browser. The complexity comes in the how of that process and various other cool tricks such as hot reloading (creating a new bundle each time you save changes to a file) and tree shaking (eliminating unused code).

The first step in creating the bundler will be creating a command line tool so we can use npm scripts to initiate everything. Our framework aprender already has a package.json file so we begin by adding the following command.

{
  "name": "aprender",
  "version": "1.0.0",
  "description": "",
  "main": "",
  "scripts": {
    "test": "node tests",
    "demo": "maleta demo/index.html --entry index.js"
  }
}

At this point it is worth exploring what happens when we type npm run demo in the terminal. Before running the commend, we first create a symlink between aprender and our build tool, which will be called maleta. The symlink is created by:

Creating a folder called maleta on the same folder level as aprender
In the terminal, navigate to maleta and type npm link
Navigate to aprender and type npm link maleta

When npm run demo is executed, npm grabs the scripts object in aprender's package.json file and runs whatever command is assigned to the property demo. The first part of the demo command is referencing maleta, our module bundler. npm will process maleta's package.json file and look for an object called bin. It looks like this:

"bin": {
  "maleta": "bin/cli.js"
}

The bin folder in any npm package contains executable files. The value of maleta is the path to the file cli.js, which contains the following code:

#!/usr/bin/env node

const program = require('commander');
const version = require('../package.json').version;
const bundler = require('../src/bundler');

program.version(version);

program
  .command('serve <filename>')
  .description('serves the files')
  .option(
    '--entry <file>',
    'set the name of the entry JS file'
  )
  .action(bundle);

program
  .command('help [command]')
  .description('display help information for a command')
  .action(function(command) {
    let cmd = program.commands.find(c => c.name() === command) || program;
    cmd.help();
  });

const args = process.argv;

// Make serve the default command except for --help
if (args[2] === '--help' || args[2] === '-h') args[2] = 'help';
if (!args[2] || !program.commands.some(c => c.name() === args[2])) args.splice(2, 0, 'serve');

program.parse(process.argv);

function bundle (entryJsFile, command) {
  bundler(entryJsFile, {
    entryJsFile: command.entry
  });
}

This file is executed by your operating system's shell. npm does this by using node's child_process method. The shebang #!/usr/bin/env node at the top of the file tells your operating system which interpreter or application to use when executing the file (if you are using Windows, this will be slightly different). When the node process is launched, any arguments specified are passed to the process.argv property. The first two arguments refer to the absolute pathname of the executable that started the process and the path to the JavaScript file being executed. Every argument from index two onwards is used by whatever code is being executed.

Maleta's CLI tool is built using commander. Commander exposes an object with a number of methods. We can use the version method to return the bundler version by typing maleta -V or maleta --version. After that we use the command method to begin creating our commands. command takes one argument written in the following syntax command <requiredArg> [optionalArg]. Our CLI tool has two commands - one to serve the app and another to print help text. The string specified via description is displayed when a user runs the help command. The action method is used to specifiy the callback function which runs when the command is executed. It receives the argument(s) passed in via the <> or [] brackets and the commander object, which will have the names of any specified options among its properties.

Taking inspiration from Parcel, we make serve the default argument if no argument has been passed and then use commander's parse method to add the arguments to the commander object. Finally, bundle calls the imported bundler function with the entry file.

The bundler at work

Maleta borrows much of its structure from Minipack, a similar project written by Ronen Amiel that explains how bundlers work. The only differences are that Maleta bundles both ES and CommonJS modules, has a CLI tool and spins up a server to run the app. At the core of our bundler's work is the dependancy graph. This lists all the files used in an application along with any dependencies. Before building that graph, we will use the entry file to create a rootAsset object with the following structure:

const rootAsset = {
  outDir: '', // the path of the directory where the bundle will created
  content: '', // the code in the file
  entryJsFilePath: '', // the path of the entry JavaScript file
  rootDir: '', // the path of the directory where the entry file lives
  dependencyGraph: '', // the dependencies of the entry file
  ast: '' // an abstract syntax tree created from the code in the file
}

Bundlers should be able to handle JavaScript or HTML files as the entry file but for simplicity, Maleta will only accept HTML files as the starting point. The function which creates the rootAsset object is:

function createRootAssetFromEntryFile(file, config) {
  rootAsset.content = fs.readFileSync(file, 'utf-8');
  rootAsset.rootDir = getRootDir(file);
  rootAsset.outDir = path.resolve('dist');

  if (config.entryJsFile) {
    rootAsset.ast = htmlParser(rootAsset.content);
    rootAsset.entryJsFilePath = path.resolve(rootAsset.rootDir, config.entryJsFile);
  } else {
    extractEntryJSFilePathFromEntryFile(rootAsset);
  }

  rootAsset.dependencyGraph = createDependencyGraph(rootAsset.entryJsFilePath);

  return rootAsset;
}

It receives the arguments passed into the bundler function by the CLI tool. The only interesting activities occur in the htmlParser, extractEntryJSFilePathFromEntryFile and createDependencyGraph functions. fs and path are node modules which are documented here and getRootDir does what its name states. Note: Reading the file synchronously with fs.readFileSync is not very performant as it a blocking call but we are not too worried about that at this moment.

When we call htmlParser it receives the following content from our demo app:

<html>
  <head>
    <title>Hello, World</title>
  </head>
  <body>
    <div id="app"></div>
    <script src="./index.js"></script>
  </body>
</html>

htmlParser refers to the module posthtml-parser, a tool for parsing and turning HTML into an abstract syntax tree (AST). Our npm command demo: maleta demo/index.html --entry index.js helps us easily find the path to related entry JavaScript file. However, if the --entry option is missing, we call extractEntryJSFilePathFromEntryFile.

function extractEntryJSFilePathFromEntryFile(rootAsset) {
  const parsedHTML = htmlParser(rootAsset.content);

  rootAsset.ast = parsedHTML;
  parsedHTML.walk = walk;

  parsedHTML.walk(node => {
    if (node.tag === 'script') {
      if (node.attrs.src.endsWith('/index.js')) {
        rootAsset.entryJsFilePath = path.resolve(rootAsset.rootDir, node.attrs.src)
      }
    }

    return node;
  });

  if (!rootAsset.entryJsFilePath) throw Error('No JavaScript entry file has been provided or specified. Either specify an entry file or make sure the entry file is named \'index.js\'');
}

The only difference here is posthml's walk method which we have attached to the AST. We use it to traverse the tree and ensure the HTML file has a link to a JavaScript file called index.js.

Building the dependency graph

Our graph will be an array of objects listing every module in the application. Each object will have:

an id
the code from the module
the original filename
an array of the relative file paths of that module's dependencies
an object with the ids of those same dependencies.

The first thing createDependencyGraph does is create the main asset from the entry JavaScript file using this function:

function createJSAsset(filename) {
  const content = fs.readFileSync(filename, 'utf-8');
  const ast = babylon.parse(content, { sourceType: 'module' });

  const relativeFilePathsOfDependenciesArray = [];

  traverse(ast, {
    ImportDeclaration({ node }) {
      relativeFilePathsOfDependenciesArray.push(node.source.value)
    },
    CallExpression({ node })  {
      const { callee, arguments: args } = node;
      if (
        callee.name === 'require' &&
        args.length === 1 &&
        args[0].type === 'StringLiteral'

      ) {
        relativeFilePathsOfDependenciesArray.push(args[0].value)
      }
    }
  })

  const id = moduleID++;

  const { code } = transformFromAstSync(ast, null, {
    presets: ['@babel/env'],
    cwd: __dirname
  });

  return {
    id,
    code,
    filename,
    relativeFilePathsOfDependenciesArray,
    mapping: {}
  }
}

babylon is the same JavaScript parser used by babel. Its parse method runs the given code as a JS program and in the second argument you pass an options object which tells it whether it is dealing with a module or script. Its output is an AST according to the babel AST format. We use it with the babel plugin traverse (babel-traverse) to find all the dependency references. ImportDeclaration finds all the ES module imports whilst CallExpression searches for every function call expression, from which we can check if its being done with the require keyword.

The next task is to parse the JavaScript code in the file. transformFromAstSync is a method from the babel/core module and it turns our AST into the final code which will run in the browser. It also creates a source map. In the config object it is important to set the working directory to maleta otherwise any file paths will be resolved to whichever directory is running maleta, which in our case is aprender.

Once the main asset has been created from the entry JavaScript file, it is assigned to the assetQueue array for processing. This array is a queue which will eventually contain assets representing every JavaScript file in the application. The relationship between each asset and its dependencies is stored in an object called mapping. Every property on this object is the file name of each dependency along with its id.

Creating the bundle

function createBundle(entryFile, config) {
  let modules = '';
  let bundle;
  const rootAsset = createRootAssetFromEntryFile(entryFile, config);
  const bundlePath = path.resolve(rootAsset.outDir, 'index.js');
  const bundleHtml = htmlRender(rootAsset.ast);
  const bundleHtmlPath = path.resolve(rootAsset.outDir, 'index.html');

  // ...
}

createBundle is the function used by our CLI to kickstart the bundling process. createRootAssetFromEntryFile performs all the steps listed above and returns a rootAsset object. From that, we create the file paths for the output files. We also use htmlRender(which is actually posthtml-render) to turn the AST we grabbed from the entry HTML file into a new HTML tree. The next step is to iterate over the dependency graph and create the bundled code like so:

function createBundle(entryFile, config) {
  // ...

  rootAsset.dependencyGraph.forEach(mod => {
    modules += `${mod.id}: [
      function (require, module, exports) {
        ${mod.code}
      },
      ${JSON.stringify(mod.mapping)},
    ],`;
  });

  bundle = `
    (function(modules) {
      function require(id) {
        const [fn, mapping] = modules[id];

        function localRequire(name) {
          return require(mapping[name]);
        }

        const module = { exports: {} };

        fn(localRequire, module, module.exports);

        return module.exports;
      }

      require(0);
    })({${modules}})
  `;

  // ...
}

The bundle explained

The bundle is an immediately invoked function expression (IIFE), a JavaScript function that runs immediately as soon as it is defined. We assign it to the bundle variable and then pass in the modules object as the argument. Each module is an array with a function that executes code for that module as its first element and the module/dependency relationship as its second element.

The first thing the IIFE does is create a require function which takes an id as its only argument. In this function, we destructure the array and access the function and mapping object of each module. The modules will have require() calls to relative file paths and some might make calls to the same file paths even though they are referring to different dependencies. We handle that by creating a dedicated local require function which turns file paths into module ids.

For example, in our demo application the require(0) call at the end of the IIFE results in the following:

function require(id) {
  const [fn, mapping] = modules[id];
  /* the value of fn */
    function (require, module, exports) {
      "use strict";
      var aprender = require('../src/aprender');
      var button = aprender.createElement('button', {
        children: ['Click Me!']
      });
      var component = aprender.createElement('div', {
        attrs: {
          id: 'root-component'
        },
        children: ['Hello, world!', button]
      });
      var app = aprender.render(component);
      aprender.mount(app, document.getElementById('app'));
    }
  /* the value of mapping */ 
  {"../src/aprender": 1}
}

require('../src/aprender'); is really localRequire('../src/aprender'). Internally, localRequire makes this recursive call require(mapping['../src/aprender']. mapping['../src/aprender'] returns the value 1, which is the id of the entry JavaScript file's only dependency. require(1) returns:

function require(id) {
  const [fn, mapping] = modules[id];
  /* the value of fn */
    function (require, module, exports) {
      "use strict";
      var createElement = require('./createElement');
      var render = require('./render');
      var mount = require('./mount');
      module.exports = {
        createElement: createElement,
        render: render,
        mount: mount
      };
    }

  /* the value of mapping */
  {"./createElement":2,"./render":3,"./mount":4}
}

Each time the code in our dependencies makes a require call, it will be destructured in this way. The rest of the code in the bundler IIFE is:

function localRequire(name) {
  return require(mapping[name]);
}

const module = { exports: {} };

fn(localRequire, module, module.exports);

return module.exports;

localRequire wraps the recursive call we explained above and fn(localRequire, module, module.exports) executes the function we destructured at the beginning of the require function. All the exports from the dependencies of module in question will be stored in the module object. In our demo application, createElement, render and mount all export functions and an object with all these exports is value of the aprender module.

Serving the bundle

Once the bundle is ready, we create an output directory, create the index.js and index.html files for the demo application and then serve them using http and serve-static.

function createBundle(entryFile, config) {

  //...

  // create the output directory if it does not exist
  if (!fs.existsSync(rootAsset.outDir)) {
    fs.mkdirSync(rootAsset.outDir);
  }


  // create output html and js files
  fs.writeFileSync(bundlePath, bundle);
  fs.writeFileSync(bundleHtmlPath, bundleHtml);

  // create server and serve files
  const serve = serveStatic(rootAsset.outDir); 
  const server = http.createServer( function onRequest(req, res) {
    serve(req, res, finalhandler(req, res));
  });

  server.listen(3000);
  console.log(`${chalk.bold('Now serving the application on')} ${chalk.red('http://localhost:3000')}`);
}

Summary

The bundler we created is by no means perfect and no doubt contains many holes and candidates for improvement. However, it is functional and that is the most important thing. We have reached a stage in our project where we can view our application in a browser. In the next article, we will return to our UI framework and add the functionality which allow us to create more complicated demo application.

Learn JavaScript by building a UI framework: Part 3 - Rendering & testing DOM elements

Carl Mungazi — Thu, 18 Jul 2019 11:26:28 +0000

This article is the third in a series of deep dives into JavaScript. You can view previous articles here and here.

Last time out we wrote a testing library for our framework. In today's post, we will add more functionality to the framework which will turn this:

const aprender = require('../src/aprender');

const button = aprender.createElement('button', { children: ['Click Me!'] });
const component = aprender.createElement(
  'div', 
  {
    attrs: { id: 'app'},
    children: [
      'Hello, world!',
      button
    ]
  },
);

const app = aprender.render(component);

aprender.mount(app, document.getElementById('app'));

into this:

We will also new tests:

let element;
let $root;
let app;

beforeAll(() => {
  element = createElement('div', {
    children: [
      createElement('h1', { children: ['Hello, World'] }),
      createElement('button', { children: ['Click Me!'] }),
    ]
  });

  createMockDom();

  $root = document.createElement("div");
  $root.setAttribute('id', 'root');
  document.body.appendChild($root);

  app = render(element);
});


check('it creates DOM elements', () => {
  assert.isDomElement( app );
});

check('it mounts DOM elements', () => {
  mount(app, document.getElementById('root'));

  assert.isMounted(app, $root);
});

From JavaScript object to DOM element

Let us begin by reminding ourselves of the current file structure:

- aprender
  - src
    - createElement.js
  - tests
    - index.js
- examinar
  - node_modules
    - colors
  - package.json
  - package-lock.json
  - src
    - assertions
      - deep-equal.js
      - index.js
    - index.js

Aprender is the name of our framework and it houses the createElement function and its associated test. Examinar is our testing framework and it has a node modules folder containing the colors package. In the assertions folder, the object equality utility function deep-equal sits in its own file. The index file contains the assertions isDeeplyEqual and throws. The first thing we will do is create a folder called demo in aprender so we can build the example application that prints Hello, World to the browser. In the demo folder we create an index.html and index.js file:

<html>
  <head>
    <title>Hello, World</title>
  </head>
  <body>
    <div id="app"></div>
    <script src="./index.js"></script>
  </body>
</html>

const aprender = require('../src/aprender');

const button = aprender.createElement('button', { children: ['Click Me!'] });
const component = aprender.createElement(
  'div', 
  {
    attrs: { id: 'app'},
    children: [
      'Hello, world!',
      button
    ]
  },
);

const app = aprender.render(component);

aprender.mount(app, document.getElementById('app'));

The index.js file contains two new methods: render and mount. render turns the JavaScript object assigned to component into a DOM element whilst mount appends the newly created element to <div id="app"></div>. The code for render is:

function renderElement({ type, attrs, children }) {
  const $el = document.createElement(type);

  for (const [attribute, value] of Object.entries(attrs)) {
    $el.setAttribute(attribute, value);
  }
  for (const child of children) {
    $el.appendChild(render(child));
  }

  return $el;
};

function render(vNode) {
  if (typeof vNode === 'string') {
    return document.createTextNode(vNode);
  }

  return renderElement(vNode);
};

Since render can be called recursively via renderElement, it needs to first check if it is dealing with a string. If it is, we create a text node and return it. Otherwise, we return the result of calling renderElement with our virtual dom object. renderElement then loops through the attrs object, sets any attributes it finds and then works on any children present in the children array. The mount function is more straightforward:

function mount($app, $root) {
  return $root.appendChild($app);
}

If we were building React, for example, mount's equivalent would be ReactDOM.render. Like ReactDOM.render, mount takes the element we want to add to the DOM as the first argument and then appends it to the element specified as the second argument. Of course, React does a lot more than that but between createElement, render and mount, you have the basic workflow used by most UI frameworks to create DOM elements. The differences lie in all the things they do in between to manage events, keep track of state and perform updates.

Aprender is not a framework destined for production but if it were, we would need to ensure that it correctly creates and appends DOM elements. This testing will happen in node, which has no DOM, so what can we do? We could:

Use jsdom. Jest and Enzyme, two popular testing tools, use it
Stub DOM objects as shown here
Use a headless browser

All of these are good options but for us, using jsdom or a headless browser is overkill. Stubbing DOM objects does not fully meet our requirements because we want to simulate the process of creating a virtual dom object, turning it into a DOM element and then adding it to the DOM tree. Fortunately for us, the framework Mithril.js has already tackled this problem. The maintainers have mocked the DOM in node as part of their testing suite. So we will model our own DOM on their implementation but adapt it to suit our needs. Those needs are expressed in the following tests:

group('aprender', () => {
  let element;
  let $root;
  let app;

  beforeAll(() => {
    element = createElement('div', {
      children: [
        createElement('h1', { children: ['Hello, World'] }),
        createElement('button', { children: ['Click Me!'] }),
      ]
    });

    createMockDom();

    $root = document.createElement("div");
    $root.setAttribute('id', 'root');
    document.body.appendChild($root);

    app = render(element);
  });

  check('it creates DOM elements', () => {
    assert.isDomElement(app);
  });

  check('it mounts DOM elements', () => {
    mount(app, document.getElementById('root'));

    assert.isMounted(app, $root);
  });
});

beforeAll is a hook which takes a function as its argument and then calls it. The function we provided creates our virtual dom object and then initialises our mock DOM via createMockDom().

What is the DOM?

According to MDN Web Docs, "The Document Object Model (DOM) is the data representation of the objects that comprise the structure and content of a document on the web". Our new tests dictate that we need the createElement, appendChild, setAttribute, createTextNode and getElementById methods, so the skeleton of our mock DOM object looks like this:

const document = {
  createElement() {
    appendChild,
    setAttribute
  },
  createTextNode() {},
  getElementById() {}
}

function appendChild() {}
function setAttribute() {}

We will begin by fleshing out the createElement method:

createElement(tag) {
  return {
    nodeType: 1,
    nodeName: tag.toUpperCase(),
    parentNode: null,
    childNodes: [],
    appendChild,
    setAttribute,
    attributes: {},
    $$dom: true
  }
}

It returns an object which represents a DOM element. Real DOM elements contain more properties than listed above but we are only implementing the ones we need. The $$dom property is our own creation and we will look at why we need it when we cover the new test assertions isMounted and isDomElement. The next thing we will do is add the functionality for setting attributes:

function setAttribute(name, value) {
  this.attributes[name] = value;
}

And appending children:

function appendChild(child) {
  let ancestor = this;

  if (ancestor === child) throw new Error("Child element cannot be equal to parent element");
  if (child.nodeType == null) throw new Error("The child is not a DOM element");

  const index = this.childNodes.indexOf(child);
  if (index > -1 ) this.childNodes.splice(index, 1);

  this.childNodes.push(child);
}

The most interesting part of this method is that if the child element already exists in the childNodes array, we remove it and re-insert it at the end of the array. This behaviour mimics what happens in the browser if you append a child which already exists on the target element.

Next we move on to the createTextNode method:

createTextNode(text) {
  return {
    nodeType: 3,
    nodeName: "#text",
    parentNode: null,
    data: text
  }
}

There is an interesting Stack Overflow answer explaining the differences between the data property we are using and the other properties which exist on text nodes and can also hold the text value.

After createTextNode, we come to the getElementById function. There is no definitive answer on how different browsers have implemented this particular method but from reading the HTML specification, we can see that traversing the DOM tree is a viable option. Our DOM tree will not be large enough to need traversing so we will opt for the simpler option of creating a new property called _elementIds on our document object and assigning it an empty object. We will populate this object in the setAttribute method every time an id is set:

function setAttribute(name, value) {
  this.attributes[name] = value;

  if (name === 'id') {
    if (document._elementIds[value]) {
      throw new Error(`${value} is already the id of an existing element`);
    }
    document._elementIds[value] = this;
  }
}

When called, getElementById will return the element if its id exits in _elementIds.

getElementById(id) {
  if (document._elementIds[id]) {
    return document._elementIds[id]
  }

  return null;
}

Testing the DOM

Our first new assertion isDomElement tries to answer the following question - how can you tell if something is a DOM element? The answer is not straightforward. We can attempt an answer by taking inspiration from how React adds a $$typeof property to React elements to help distinguish them from anything else not created by React. We will appropriate this by creating a property called $$dom on our document object and assigning it the value true. Then we write isDomElement:

isDomElement(element) {
  if (element.hasOwnProperty("$$dom") && element.$$dom) return true;

  throw new Error('The supplied element is not a DOM element')
}

Our implementations of $$dom and _elementIds are not the best ways of adding what is effectively meta data to our document object. For one, we could have used Symbols to ensure those properties do not show up via iteration methods such as Object.keys or for..of. But even then, symbols can still be found through the Reflect.ownKeys or Object.getOwnPropertySymbols methods so that solution is not perfect. For getElementById we could traverse the DOM tree and find the element that way. Thankfully, our needs at this stage are relatively simple.

The second assertion, isMounted, tests the results of calling the mount method. As mentioned earlier, mount plays a similar role to ReactDOM.render, so all we need to do is check that our specified DOM element is the only child of the <div id="app"></div> element we created in the index.html file.

isMounted(element, parentElement) {
  if (parentElement.childNodes.length > 1) throw new Error('The root element has more than one child');

  if (parentElement.childNodes[0] === element) return true;

  throw new Error('The supplied element has not been mounted');
}

All the new tests are predicated on the existence of a DOM like structure in the testing environment. This is handled by the createMockDom method, which first creates the root element that is supposed to exist on every non-empty HTML document. It then adds a head and body element as children. Strictly speaking, the html element should also exist in a childNodes array on the document object but we can skip this in our implementation. We then finish by adding our document object to the node global variable.

function createMockDom() {
  document.documentElement = document.createElement("html");
  document.documentElement.appendChild(document.createElement("head"));
  document.body = document.createElement("body");
  document.documentElement.appendChild(document.body);

  global.document = document;
}

Summary

We now have a framework which creates and renders DOM elements. Or do we? Our tests pass but how can we view our application in the browser? We will cover that in the next article by building a module bundler and setting up our development environment. In the meantime, you can explore the code we have written so far here.

Resources

Learn JS by building a UI framework: Part 2 - Testing

Carl Mungazi — Fri, 14 Jun 2019 18:08:49 +0000

This article is the second in a series of deep dives into JavaScript. You van view the first article here.

This series does not comprehensively cover every JavaScript language feature. Instead, features are covered as they crop up in solutions to problems. Also, every post is based on tutorials and open source libraries produced by other developers, so like you, I too am also learning new things with each article.

Let the testing begin

To test or not to test, that is the question. At the end of the last post we pondered our next move after writing the first piece of code for our framework. We surmised that we could either write more code until we have a working framework or begin adding tests. Each option has tradeoffs. Writing more code means quickly creating a working prototype. Alternatively, writing tests means more robust code early on but at the price of slower development. There is no right or wrong answer. Sometimes you need to rapidly prototype and test different solutions whilst other times you add tests so you write better code.

We will take the testing route because in addition to making our code more robust, it gives us another chance to explore JavaScript from a different angle. Below is a reminder of what our createElement function currently looks like:

function createElement (type, opts) {
  if (type == null || typeof type !== 'string') {
    throw Error('The element type must be a string');
  }

  if (arguments[1] !== undefined && Object.prototype.toString.call(opts) !== '[object Object]') { 
    throw Error('The options argument must be an object'); 
  }

  const { attrs = {}, children = [] } = opts || {};

  return {
    type,
    attrs,
    children
  }
}

And below are the tests we will cover in this article.

group('createElement function', () => {
  check('it creates a virtual dom object', () => {
    const target = createElement('div', {});
    const copy = { type: 'div', attrs: {}, children: [] };
    assert.isDeeplyEqual(target, copy);
  }); 

  check('it throws errors when a string is not specified as the first argument', () => {
    const err = () => createElement(1, null);
    assert.throws(err, 'The element type must be a string');
  });

  check('it throws errors when the options argument is not an object', () => {
    const err = () => createElement('h1', null);
    assert.throws(err, 'The options argument must be an object');
  });

  xcheck('it creates DOM elements', () => {
    // do some testing
  });

  xcheck('it mounts DOM elements', () => {
    // do some testing
  });
});

When the tests run, we will see this:

The work in the tests is being done by group, check, xcheck and assert. We will begin by examining group and check. Note: some code has been omitted for brevity

function group (title, fn) {
  console.log(title);
  fn();
}

function check(title, fn) {
  console.log(title);
  try {
    fn();
  } catch (e) {
    console.log(title);
    console.log((e.message);
    console.log(e.stack);
  }
};

The behaviour of both methods is simple but they introduce two new things we have not covered before: try/catch and arrow functions.

In the last post we had our first foray into error handling via the throw keyword. Try/catch is another error handling feature. It is used to test a block of code for errors and then handle any exceptions. In the check function, the try block will execute the function that has been passed in as the second argument. Any execptions are sent to the catch block, which is passed an error object. This object contains a message describing the error and a stack trace indicating which line of code caused it. Note: the variable e passed to the catch block is block scoped, so this code would result in an error:

function errorWaitingToHappen() {
  try {
    throw Error('I am an error');
  } catch (errorWithLongVariableName) {
    console.log(errorWithLongVariableName)
  }

  console.log(errorWithLongVariableName) // this is undefined
}

errorWaitingToHappen(); // Uncaught ReferenceError: errorWithLongVariableName is not defined

Arrow functions were introduced in ES6 and generally speaking, they behave like normal functions except in these cases. We are using them because none of the issues outlined in the links are applicable and they make our code terser.

Our next lesson comes not from the testing library we are building but from the code being tested. createElement should throw exceptions when called with certain arguments. Testing this behaviour requires us to pass it to assert.throws with those arguments specified. We could do it like this:

assert.throws(createElement, ['h1', null], 'The options argument must be an object');

And then implement assert.throws:

throws(fn, args, errMsg = '') {
  try {
    fn(...args);
  } catch (e) {
    // do something
  }
}

By passing in the error inducing arguments in an array, we can spread them into createElement to successfully trigger the exception. However, this is not an optimal solution because it pollutes our assert.throws implementation. That function should not care about the arguments needed to throw an exception. Its sole purpose is to execute the function and check for errors. We can achieve that goal by introducing another new concept: lazy evaluation:

const err = () => createElement('h1', null);
assert.throws(err, 'The options argument must be an object');

Even though we are calling createElement in the function assigned to err, we need not worry because the actual execution happens within the context of assert.throws. It is only evaluated when needed, hence the term. Had we written this:

assert.throws(createElement('h1', null), 'The options argument must be an object');

createElement will be called in the context of check. And because check has its own try/catch block, it would handle the exception, meaning that assert.throws would never run and our test would fail when it should have passed. With that sorted, we can fill out the rest of assert.throws.

throws(fn, errMsg = '') {
  const didNotThrowErr = new Error('The supplied function didn\'t throw an error');

  try {
    fn();
    throw didNotThrowErr;
  } catch (e) {
    if (e === didNotThrowErr) throw didNotThrowErr;

    if (!errMsg || e.message === errMsg) return true;

    throw new Error(`\n\nFound: ${e.message}\nWanted: ${errMsg}\n\n`); 
  }
}

Checking for object equality

The implementation of assert.isDeeplyEqual raises some interesting questions. As we can see below, its purpose is to check if any two given objects are equal.

check('it creates a virtual dom object', () => {
  const target = createElement('div', {});
  const copy = { type: 'div', attrs: {}, children: [] };
  assert.isDeeplyEqual(target, copy);
});

What kind of equality are we checking for here? Compound values (objects, arrays etc) in JavaScript are assigned by reference, so using the in-built equality operators will not help us. We want to ensure both objects contain the same properties and that those properties also contain the same values. Here are some questions our solution needs to answer:

Are both values objects?
Do they contain the same number of properties?
Do all those properties hold the same value? If they do:
- Are they primitive values we can easily check using equality operators?
- If they are compound values, what do we do?
- Are we going to handle every type of compound value?

We will call the function which does the checks deepEqual and the first thing we need to do is check if we are dealing with objects. This check will be repeated again later so we can abstract it away with the following utility function:

function getLengthAndType(obj) {
  if (Object.prototype.toString.call(obj) === '[object Array]') {
    return { type: "array", length: obj.length }
  }

  if (Object.prototype.toString.call(obj) === '[object Object]') {
    return { type: "object", length: Object.keys(obj).length }
  }

  return null;
}

We return an object with useful information we will use later and it also helps us avoid repeating the Object.prototype.toString.call call. Here is the utility in action in the first part.

function deepEqual(obj, comparisonObj) {
  const objInfo = getLengthAndType(obj);
  const comparisonObjInfo = getLengthAndType(comparisonObj); 

  // only go forward with arrays or objects
  if ( !objInfo || !comparisonObjInfo) {
    return false
  }

  if (objInfo.length !== comparisonObjInfo.length || objInfo.type !== comparisonObjInfo.type) {
    return false
  }

getLengthAndType returns null for non-arrays and non-objects, so we can quickly establish the type of values we are comparing since null is a falsy value. After that, we check the length and ensure both objects are the same type. The next thing to consider is how we are going to iterate over our array or object, and check each value.

if (objInfo.type === 'array') {
  for (var i = 0; i < objInfo.length; i++) {
    if (compare(obj[i], comparisonObj[i]) === false) return false;
  }    
} else {
  for (let [key] of Object.entries(obj)) {
    if (compare(obj[key], comparisonObj[key]) === false) return false;
  } 
}

Using a for loop allow us to iterate over the array and check the values easily. However, this solution does have one drawback in that it assumes the values we want to check share same index position in both arrays. This is fine for our use case because we are checking that the objects defined in the children array of our virtual dom object are in the same position. Object.entries returns an array of an object's [key, value] pairs. Coupled with for..of, which creates a loop over iterable objects, we can also iterate over an object. An alternative approach would have been to use for..in but this would require an additional hasOwnProperty check because for..in also iterates over inherited properties, something Object.entries does not do. This alternative approach would look like this:

if (objInfo.type === 'array') {
  for (var i = 0; i < objInfo.length; i++) {
    if (compare(obj[i], comparisonObj[i]) === false) return false;
  }    
} else {
  for (var prop in obj ) {
    if (obj.hasOwnProperty(prop)) {
      if (compare(obj[prop], comparisonObj[prop]) === false) return false;
    }
  }  
}

The most interesting thing about deepEqual is the concept it introduces in its internal compare function. As you can see below, we easily compare functions and primitive values with the toString() method and equality operators, respectively. However, comparing arrays or objects is more complicated.

const compare = (val, comparisonVal) => {
  const isArrayOrObject = getLengthAndType(val);
  const isFunction = Object.prototype.toString.call(val) === '[object Function]';

  if (isArrayOrObject) {
    if (!deepEqual(val, comparisonVal)) return false;
  } 

  else {
    if (isFunction) {        
      if (val.toString() !== comparisonVal.toString()) return false;
    } else {
      if (val !== comparisonVal) return false;
    }
  }
};

Up until now, we have been declaring our functions in one place and then calling them in another. With deepEqual, we are calling it within itself. This process is called recursion and makes deepEqual a recursive function. Given this call isDeeplyEqual( {arr: [1, 2, 3]}, {arr: [1, 2, 3]} ), when the program reaches the compare function, this happens:

// The initial recursive call
const compare = (val, comparisonVal) => {
  // val is {arr: [1, 2, 3]}
  // comparisonVal is {arr: [1, 2, 3]}
  const isArrayOrObject = getLengthAndType(val); // { type: "object", length: 2 }

  if (isArrayOrObject) { // true
    if (!deepEqual(val, comparisonVal)) return false; // recursion! 
  }
  //...
}

// We reach compare again and make another recursive call
const compare = (val, comparisonVal) => {
  // val is [1, 2, 3]
  // comparisonVal is [1, 2, 3]
  const isArrayOrObject = getLengthAndType(val); // { type: "array", length: 3 }

  if (isArrayOrObject) { // true
    if (!deepEqual(val, comparisonVal)) return false; // more recursion! 
  }
  //...
}

// No more recursive calls 
// We are now comparing every element in the array [1, 2, 3]
const compare = (val, comparisonVal) => {
  // val is 1
  // comparisonVal is 1
  const isArrayOrObject = getLengthAndType(val); // false

  if (isArrayOrObject) { // false
    if (!deepEqual(val, comparisonVal)) return false; // no recursion :-(
  }
  //...
}

Recursion is a good way to navigate and perform operations on nested data structures. In our scenario it abstracts away the complexity of writing conditional checks to handle what will be an unknown number of nested objects and arrays. Each time we call a function, a new execution context is created and added to the call stack, so we can offload that work to the JavaScript engine and wait for whatever is returned. Put together, deepEqual looks like this:

function getLengthAndType(obj) {
  if (Object.prototype.toString.call(obj) === '[object Array]') {
    return { type: "array", length: obj.length }
  }

  if (Object.prototype.toString.call(obj) === '[object Object]') {
    return { type: "object", length: Object.keys(obj).length }
  }

  return null;
}

function deepEqual(obj, comparisonObj) {
  const objInfo = getLengthAndType(obj);
  const comparisonObjInfo = getLengthAndType(comparisonObj); 

  // only go forward with arrays or objects
  if ( !objInfo || !comparisonObjInfo) {
    return false
  }

  if (objInfo.length !== comparisonObjInfo.length || objInfo.type !== comparisonObjInfo.type) {
    return false
  }

  const compare = (val, comparisonVal) => {
    const isArrayOrObject = getLengthAndType(val);
    const isFunction = Object.prototype.toString.call(val) === '[object Function]';

    if (isArrayOrObject) {
      if (!deepEqual(val, comparisonVal)) return false;
    } 

    else {
      if (isFunction) {        
        if (val.toString() !== comparisonVal.toString()) return false;
      } else {
        if (val !== comparisonVal) return false; // we are comparing primitive values
      }
    }
  };

  if (objInfo.type === 'array') {
    for (var i = 0; i < objInfo.length; i++) {
      if (compare(obj[i], comparisonObj[i]) === false) return false;
    }    
  } else {
    for (let [key] of Object.entries(obj)) {
      if (compare(obj[key], comparisonObj[key]) === false) return false;
    }
  }

  return true; // nothing failed
}

So far we have only looked at the code responsible for the actual testing but how do we show the test results and other information to the user? Unlike our framework which will be used to create user interfaces in the browser, our testing framework only works on the command line. Let us begin with the final implementations of check and group. xcheck is also included but it is not doing much.

const colors = require('colors');
const assert = require('./assertions');

const repeat = (str, n) => Array(n).join(str);
const indent = n => repeat('    ', n);
const indentLines = (str, n) => indent(n) + str.replace(/\n/g, `\n${indent(n)}`);
const log = str => console.log(str);
const summary = { success: 0, fail: 0, disabled: 0 };

let indentLevel = 0;
let examinar;

function group(title, fn) {
  indentLevel++;
  log(`\n${indent(indentLevel)}⇨ ${title}`.yellow);
  fn();
  indentLevel--;
}

function check(title, fn) {
  try {
    fn();
    log(`${indent(indentLevel + 1)}${' OK '.bgGreen.black} ${title.green}`);
    summary.success++;
  } catch (e) {
    log(`${indent(indentLevel + 1)}${' FAIL '.bgRed.black} ${title.red}`);
    log(indentLines(e.message.red, indentLevel + 1));
    log(indentLines(e.stack.red, indentLevel + 1));
    summary.fail++;
  }
}

function xcheck(title) {
  log(`${indent(indentLevel + 1)}${' DISABLED '.bgWhite.black} ${title.gray}`);
  summary.disabled++;
}

function end() {
  log(`\n${repeat('.', 60)}\n`);
  log('Test summary:\n');
  log(`    Success: ${summary.success}`.green);
  log(`    Fail: ${summary.fail}`.red);
  log(`    Disabled: ${summary.disabled}\n\n`.gray);

  if (summary.fail > 0 ) process.exit(1);
  process.exit(0);
}

module.exports = { assert, check, end, group, xcheck };

Displaying the results and relevant information is achieved through a combination of indentation, adding colour and summarisation. Each time we group some tests, we log the title using template literals and the color module. You will notice that this is the first time in our project we are using a third party library. This is because as much as we are trying to write our own code, that is not always possible and there are instances where, because of time or complexity, it makes more sense to leverage existing solutions. However, when using third-party code it does help to understand a high level what exactly is being abstracted away. The color module works by extending String.prototype and defining getters which return the specified colour. You can test this yourself by importing the module and writing console.log(''.__proto__). Template literals can evaluate expressions, so combined with the colour module, they create nice visuals for our tests.

The end function terminates our suite of tests after logging a summary. This is achieved by using the globally available process.exit() method which instructs Node.js to terminate the current process synchronously with an exit code: 1 for failure and 0 for success.

There is one more thing we can learn from the code at end of our function: exporting modules. We will visit this topic in greater depth in our next post when set up our development environment but for now, we can briefly cover it. Modules are units of code, much like lego pieces, which can be put together to create various things. Node treats each file as a separate module. You import files using the require keyword and export with module.exports or exports. Both module.exports or exports reference the same object so:

module.exports = { assert, check, end, group, xcheck };

is equivalent to:

exports.check = check();
exports.end = end();
exports.assert = assert;
exports.group = group();
exports.xcheck = xcheck()

Summary

In this post we made no progress with the actual framework code but we began laying a foundation for future development. In creating a testing framework we learnt about arrow functions, try/catch, lazy evaluation, recursion, template strings and exporting modules. All these lessons were brief but powerful nonetheless because we did them in the context of a very hands on project. This approach makes the concepts more concrete.

Resources

Testing Tutorial: https://medium.freecodecamp.org/learnbydiy-how-to-create-a-javascript-unit-testing-framework-from-scratch-c94e0ba1c57a

Object Equality Tutorial: https://gomakethings.com/check-if-two-arrays-or-objects-are-equal-with-javascript

Learning JavaScript by building a UI framework from scratch

Carl Mungazi — Wed, 01 May 2019 15:48:49 +0000

In my previous post I explained how APIs from your favourite libraries and frameworks can be turned into programming lessons. Today I will develop that idea by taking it a step further. We will not read other people's code. Instead, we will write our own and build a very basic and rudimentary UI framework.

Building a framework is a good way to deepen your JavaScript and programming knowledge because it forces you to explore language features and software engineering principles. For example, all web frameworks try to solve the problem of keeping application data in sync with the UI. All the solutions to this problems can touch different areas such as routing, DOM manipulation, state management and asynchronous programming.

One of the more popular ways of solving this UI-state-sync problem is using a concept known as the virtual DOM (or vdom). Instead of manipulating the DOM directly in response to data changes, we can use JavaScript objects because they are computationally much cheaper to manipulate. The vdom approach can be broken down like so:

When your application is first loaded, create a tree of JavaScript objects which describe your UI
Turn these objects into DOM elements using DOM APIs such as document.createElement
When you need to make a change to the DOM (either in response to user interaction, animations or network requests), create another tree of JavaScript objects describing your new UI
Compare the old and new tree of JavaScript objects to see which DOM elements have been changed and how
Make changes to the DOM only in places that have changed

One of the fundamental pieces of any vdom implementation is the function which creates the object. Essentially, this function must return an object containing the information needed to create a DOM element. For instance, in order to create this DOM structure:

<ul class="list">
    <li class="list-item" style="color: red;">Item 1</li>
    <li class="list-item" style="color: blue;">Item 2</li>
</ul>

You need to know the following information for each DOM element:

type of element
list of attributes
if it has any children (for each child, we also need to know the same information listed above)

This leads us to our first lesson: data structures. As Linus Torvalds said, "Bad programmers worry about the code. Good programmers worry about data structures and their relationships". So how can we represent the DOM structure above in code?

{
  type: 'ul',
  attrs: {
      'class': 'list'
  },
  children: [
    {
      type: 'li',
      attrs: {
        class: 'list-item',
        style: {
          color: 'red'
        }
      },
    },
    {
      type: 'li',
      attrs: {
        class: 'list-item',
        style: {
          color: 'blue'
        }
      },
    } 
  ]
}

We have an object with three properties and each property is either a string, object or array. How did we choose these data types?

All HTML elements can be represented by a string
HTML attributes have a key: value relationship which lends itself nicely to an object
HTML child nodes can come in a list format and creating them requires performing the same operation on each item in the list. Arrays are perfect for this

So now we know what our data structure looks like, we can move on to the function which creates this object. Judging by our output, the simplest thing to do would be to create a function with takes three arguments.

createElement (type, attrs, children) {
  return {
    type: type,
    attrs: attrs,
    children: children
  }
}

We have our function but what happens if when invoked, it does not receive all the arguments? Furthermore, does the creation of our object require every argument to be present?

This leads us to the next lesson: error handling, default parameters, destructuring and property shorthands.

Firstly, you cannot create a HTML element without specifying a type, so we need to guard against this. For errors, we can borrow Mithril's approach of throwing an error. Alternatively, we can define custom errors as described here.

createElement (type, attrs, children) {
  if (type == null || typeof type !== 'string') {
    throw Error('The element type must be a string');
  }

  return {
    type: type,
    attrs: attrs,
    children: children
  }
}

We will revisit this check type == null || typeof type !== 'string' later on but for now, let us focus on creating our object. Whilst we cannot create HTML elements without specifying a type, we can create HTML elements that have no children or attributes.

In JavaScript, if you call a function without providing any of the required arguments, those arguments are assigned the value undefined by default. So attrs and children will be undefined if not specified by the user. We do not want that because, as we will see later, the rest of our code expects those arguments to contain a value. To solve this, we will assign attrs and children default values:

createElement (type, attrs = {}, children = []) {
  if (type == null || typeof type !== 'string') {
    throw Error('The element type must be a string');
  }

  return {
    type: type
    attrs: attr,
    children: children
  }
}

As mentioned earlier, HTML elements can be created without any children or attributes, so instead of requiring three arguments in our function, we can require two:

createElement (type, opts) {
  if (type == null || typeof type !== 'string') {
    throw Error('The element type must be a string');
  }

  return {
    type: type
    attrs: opts.attr,
    children: opts.children
  }
}

We have lost the default parameters introduced earlier but we can bring them back with destructuring. Destructuring allows us to unpack object properties (or array values) and use them as distinct variables. We can combine this with shorthand properties to make our code less verbose.

createElement (type, { attrs = {}, children = [] }) {
  if (type == null || typeof type !== 'string') {
    throw Error('The element type must be a string');
  }

  return {
    type,
    attrs,
    children
  }
}

Our function can create virtual dom objects but we are not done yet. Earlier we skipped over this bit of code type == null || typeof type !== 'string'. We can now revisit it and learn something else: coercion.

There are four things to observe here:

the behaviour of the == loose equality operator
the behaviour of the || operator
the behaviour of typeof operator
the behaviour of !== operator

When I first learnt JavaScript I came across numerous articles advising against using the loose equality operator. This is because it produces surprising results such as:

1 == '1' // true
null == undefined // true

It is surprising because in the examples above, we are comparing values of four different primitive types: number, string, null and undefined. The checks above evaluate to true because == performs a coercion when comparing values of differing types. The rules that govern how this happens can be found here. For our specific case, we need to know the spec states that null == undefined will always return true. Also, !== works by performing the same checks performed by === and then negating the result. You can read the rules about that here.

Returning to our function, the first thing this type == null || typeof type !== 'string' is checking is if a null or undefined value has been passed. Should this be true, the || operator will return the result of typeof type !== 'string'. The order of how this happens is important. The || operator does not return a boolean value. It returns the value of one of the two expressions. It first performs a boolean test on type == null, which will either be true or false. If the test returns true, our error would be thrown.

However, if false is returned, || returns the value of the second expression, which in our case will either be true or false. If our check had been type == null || type and the first expression resulted in false, the second expression would return whatever value is in the variable type. The typeof operator returns a string indicating the type of the given value. We did not use it for our type == null check because typeof null returns object, which is an infamous bug in JavaScript.

With that newfound knowledge, we can take a harder look at createElement and ask ourselves the following questions:

How do we check that the second argument can be destructed?
How do we check that the second argument is an object?

Let us start by invoking our function with different argument types:

createElement (type, { attrs = {}, children = [] }) {
  if (type == null || typeof type !== 'string') {
    throw Error('The element type must be a string');
  }

  return {
    type,
    attrs,
    children
  }
}

createElement('div', []); // { type: "div", attrs: {…}, children: Array(0) }
createElement('div', function(){}); // { type: "div", attrs: {…}, children: Array(0) }
createElement('div', false); // { type: "div", attrs: {…}, children: Array(0) }
createElement('div', new Date()); // { type: "div", attrs: {…}, children: Array(0) }
createElement('div', 4); // { type: "div", attrs: {…}, children: Array(0) }

createElement('div', null); // Uncaught TypeError: Cannot destructure property `attrs` of 'undefined' or 'null'
createElement('div', undefined); // Uncaught TypeError: Cannot destructure property `attrs` of 'undefined' or 'null'

Now we modify the function:

createElement (type, opts) {
  if (type == null || typeof type !== 'string') {
    throw Error('The element type must be a string');
  }

  if (arguments[1] !== undefined && Object.prototype.toString.call(opts) !== '[object Object]') { 
    throw Error('The options argument must be an object'); 
  }

  const { attrs = {}, children = [] } = opts || {};

  return {
    type,
    attrs,
    children
  }
}

createElement('div', []); // Uncaught Error: The options argument must be an object
createElement('div', function(){}); // Uncaught Error: The options argument must be an object
createElement('div', false); // Uncaught Error: The options argument must be an object
createElement('div', new Date()); // Uncaught Error: The options argument must be an object
createElement('div', 4); // Uncaught Error: The options argument must be an object

createElement('div', null); // Uncaught Error: The options argument must be an object
createElement('div', undefined); // Uncaught Error: The options argument must be an object

Our first function was not fit for purpose because it accepted values of the wrong type. It also gave us a TypeError when invoked with null or undefined. We fixed this in our second function by introducing a new check and new lessons: error types, rest parameters and this.

When we invoked the function with null or undefined as the second argument, we saw this message: Uncaught TypeError: Cannot destructure property 'attrs' of 'undefined' or 'null'. A TypeError is an object which represents an error caused by a value not being the expected type. It is one of the more common error types along with ReferenceError and SyntaxError. This is why we reverted to using an object as our argument because there is no way of guarding against null and undefined values when destructuring function arguments.

Let us take a closer look at the check in our second iteration:

if (arguments[1] !== undefined && Object.prototype.toString.call(opts) !== '[object Object]') { 
  throw Error('The options argument must be an object'); 
}

The first question to ask is: why are we using the arguments object when rest parameters are a thing? Rest parameters were introduced in ES6 as a cleaner way of allowing developers to represent an indefinite number of arguments as an array. Had we used them, we could have written something like this:

createElement (type, ...args) {
  if (type == null || typeof type !== 'string') {
    throw Error('The element type must be a string');
  }

  if (args[0] !== undefined && Object.prototype.toString.call(args[0]) !== '[object Object]') { 
    throw Error('The options argument must be an object'); 
  }
}

This code is useful if our function had many arguments but because we are only expecting two, the former approach works better. The more exciting thing about our second function is the expression Object.prototype.toString.call(opts) !== '[object Object]'. That expression is one of the answers to the question: In JavaScript, how do you check if something is an object? The obvious solution to try first is typeof opts === "object" but as we discussed earlier, it is not a reliable check because of the JavaScript bug that returns true using typeof with null values.

Our chosen solution worked in the ES5 world by taking advantage of the internal [[Class]] property which existed on built-in objects. According to the ES5 spec, this was a string value indicating a specification defined classification of objects. It was accessible using the toString method. The spec explains toString's behaviour in-depth but essentially, it returned a string with the format [object [[Class]]] where [[Class]] was the name of the built-in object.

Most built-ins overwrite toString so we have to also use the call method. This method calls a function with a specific this binding. This is important because whenever a function is invoked, it is invoked within a specific context. JavaScript guru Kyle Simpson has outlined the four rules which determine the order of precedence for this. The second rule is that when a function is called with call, apply or bind, the this binding points at the object specified in the first argument of call, apply or bind. So Object.prototype.toString.call(opts) executes toString with the this binding pointing at whatever value is in opts.

In ES6 the [[Class]] property was removed so whilst the solution still works, its behaviour is slightly different. The spec advises against this solution, so we could seek inspiration from Lodash's way of handling this, for example. However, we will keep it because the risk of it producing erroneous results are very low.

We have created what on the surface appears to be a small and basic function but as we have experienced, the process is anything but small or basic. We can move on to the next stage but that leads to the question, what should that stage be? Our function could do with some tests but that would require creating a development workflow. Is it too early for that? If we add tests, which testing library are we going to use? Is it not better to create a working solution before doing any of this other stuff? These are the kind of tensions developers grapple with daily and we will explore those tensions (and the answers) in the next tutorial.

A journey through ReactDOM.render

Carl Mungazi — Tue, 26 Mar 2019 23:32:15 +0000

This was originally posted on Medium

ReactDOM is an object which exposes a number of top-level APIs. According to the docs, it provides ‘DOM-specific methods that can be used at the top level of your app and as an escape hatch to get outside of the React model if you need to’. One of those methods is render().

Some background notes

At the core of this re-write is a data structure called a fiber, which is an object that is mutable and holds component state and DOM. It can also be thought of as a virtual stack frame. Fiber’s architecture is split into two phases: reconciliation/render and commit. Over the course of this article we will touch upon some of its aspects but more extensive explanations can be found here:

The journey begins

ReactDOM.render() actually wraps another function and invokes it with two additional arguments, so its declaration is simple. Below is the wrapped function:

function legacyRenderSubtreeIntoContainer(parentComponent, children, container, forceHydrate, callback) {
  ...
  let root = container._reactRootContainer;
  if (!root) {
    // Initial mount
    root = container._reactRootContainer = legacyCreateRootFromDOMContainer(container, forceHydrate);
    if (typeof callback === 'function') {
      const originalCallback = callback;
      callback = function () {
        const instance = DOMRenderer.getPublicRootInstance(root._internalRoot);
        originalCallback.call(instance);
      };
    }
    // Initial mount should not be batched.
    DOMRenderer.unbatchedUpdates(() => {
      if (parentComponent != null) {
        root.legacy_renderSubtreeIntoContainer(
          parentComponent,
          children,
          callback,
        );
      } else {
        root.render(children, callback);
      }
    });
  } else {
    ...
  }
  return DOMRenderer.getPublicRootInstance(root._internalRoot);
}

The first thing React does is create the fiber tree for our application. Without this, it cannot handle any user updates or events. The tree is created by calling legacyCreateRootFromDOMContainer, which returns the following object:

{
  containerInfo: div#root
  context: null
  current: FiberNode {tag: 3, key: null, /*…*/}
  didError: false
  earliestPendingTime: 0
  earliestSuspendedTime: 0
  expirationTime: 0
  finishedWork: null
  firstBatch: null
  hydrate: false
  interactionThreadID: 1
  latestPendingTime: 0
  latestPingedTime: 0
  latestSuspendedTime: 0
  memoizedInteractions: Set(0) {}
  nextExpirationTimeToWorkOn: 0
  nextScheduledRoot: null
  pendingChildren: null
  pendingCommitExpirationTime: 0
  pendingContext: null
  pendingInteractionMap: Map(0) {}
  timeoutHandle: -1
}

This is called a fiber root object. Every React app has one or more DOM elements that act as containers and for each of these containers, this object is created. It is on this object we find a reference to our fiber tree via the current property, and its value is:

{
  actualDuration: 0
  actualStartTime: -1
  alternate: null
  child: null
  childExpirationTime: 0
  effectTag: 0
  elementType: null
  expirationTime: 0
  firstContextDependency: null
  firstEffect: null
  index: 0
  key: null
  lastEffect: null
  memoizedProps: null
  memoizedState: null
  mode: 4
  nextEffect: null
  pendingProps: null
  ref: null
  return: null
  selfBaseDuration: 0
  sibling: null
  stateNode: {current: FiberNode, containerInfo: div#root, /*…*/}
  tag: 3
  treeBaseDuration: 0
  type: null
  updateQueue: null
}

This is a fiber node and it is found at the root of every React fiber tree (this is the function which creates this object). This node is actually a special type of fiber node called a HostRoot node and it acts as a parent for the uppermost component in our application. We know it is a HostRoot node because the value of its tag property is 3 (you can find the full list of fiber node types here). Notice how at this stage, a lot of the properties, especially child, are null. We will come back to this later.

After creating the tree, React checks if we provided a callback as the third argument of the render call. If so, it grabs a reference to the root component instance of our application (in our case, it is the <h1> element) and then makes sure our callback will be called on this instance later on.

Into the thick of it

All the stuff that has happened prior to this point is preparation for the work that will render our application on screen. So far, we have a tree but it does not resemble our UI. This problem is solved with the following code:

// Initial mount should not be batched.
unbatchedUpdates(function () {
  if (parentComponent != null) {
    root.legacy_renderSubtreeIntoContainer(parentComponent, children, callback);
  } else {
    root.render(children, callback);
  }
});

Root is an object with only one property (the property _internalRoot which holds a reference to the root fiber object). It was created by calling new on the ReactRoot function, so if you look up its internal [[Prototype]] chain you will find the following method:

ReactRoot.prototype.render = function (children, callback) {
  var root = this._internalRoot;
  var work = new ReactWork();
  callback = callback === undefined ? null : callback;
  {
    warnOnInvalidCallback(callback, 'render');
  }
  if (callback !== null) {
    work.then(callback);
  }
  updateContainer(children, root, null, work._onCommit);
  return work;
}

Remember our HostRoot fiber node earlier with all the null values? In the function above, we can access it via root.current. After updateContainer() has finished its work, this is what it looks like:

{
  actualDuration: 6.800000002840534
  actualStartTime: 95592.79999999853
  alternate: FiberNode {tag: 3, key: null, /*…*/}
  child: FiberNode {tag: 5, key: null, /*…*/}
  childExpirationTime: 0
  effectTag: 32
  elementType: null
  expirationTime: 0
  firstContextDependency: null
  firstEffect: FiberNode {tag: 5, key: null, /*…*/}
  index: 0
  key: null
  lastEffect: FiberNode {tag: 5, key: null, /*…*/}
  memoizedProps: null
  memoizedState: {element: {/*…*/}}
  mode: 4
  nextEffect: null
  pendingProps: null
  ref: null
  return: null
  selfBaseDuration: 2.5999999925261363
  sibling: null
  stateNode: {current: FiberNode, containerInfo: div#root, /*…*/}
  tag: 3
  treeBaseDuration: 3.1999999919207767
  type: null
  updateQueue: {baseState: {/*…*/}, firstUpdate: null, /*…*/}
}

And the value in the child property is now a fiber node for the <h1> element:

{
  actualDuration: 4.100000005564652
  actualStartTime: 95595.49999999581
  alternate: null
  child: null
  childExpirationTime: 0
  effectTag: 0
  elementType: "h1"
  expirationTime: 0
  firstContextDependency: null
  firstEffect: null
  index: 0
  key: null
  lastEffect: null
  memoizedProps: {children: "Hello, world!"}
  memoizedState: null
  mode: 4
  nextEffect: FiberNode {tag: 3, key: null, /*…*/}
  pendingProps: {children: "Hello, world!"}
  ref: null
  return: FiberNode {tag: 3, key: null, /*…*/}
  selfBaseDuration: 0.5999999993946403
  sibling: null
  stateNode: h1
  tag: 5
  treeBaseDuration: 0.5999999993946403
  type: "h1"
  updateQueue: null
}

As you can see, the fiber tree now reflects the UI we want rendered. How did that happen? Below is a summary of the steps which took place in order to get us to this stage:

Schedule the updates

React has an internal scheduler which it uses for handling co-operative scheduling in the browser environment. This package includes a polyfill for window.requestIdleCallback, which is at the heart of how React reconciles UI updates. During the process each fiber node is given an expiration time. This is a value which represents a time in the future the work (the name given to all the activities which happen during reconciliation) being done should be completed.

This calculation involves checking if there is any pending work with a higher priority as well as the nature of the work currently going on. In our case, it is a synchronous update but in other scenarios it could be interactive or asynchronous. Examples of low priority updates include network responses or clicking a button to change colours. High priority updates include user input and animations.

Create an update queue

In our example, we only have one fiber tree and since it has no update queue, one has to be created and assigned to the updateQueue property. This queue is created with the createUpdateQueue function and it takes the fiber node's memoizedState as its argument. Memoized state refers to the state used to create the node's output. The queue is a linked list of prioritised updates and like fiber trees, it also comes in pairs. The current queue represents the visible state of the UI.

Create the work-in-progress tree and do work on it

When the work is actually being performed, React checks its internal stack to determine whether it is starting with a fresh stack or resuming previously yielded work. Soon after this check it uses a technique known as double buffering to create the workInProgress tree. React works with two fiber trees — one named current which holds the current state of the UI and another named workInProgress which reflects the future state. Every node in the current tree has a corresponding node in the workInProgress tree created from data during render.

Once the workInProgress is rendered on the screen, it becomes the new current tree. The workInProgress tree also has an update queue but it can be mutated and processed asynchronously before being rendered.

DOM element creation

The fiber node for the <h1> element is created during the work on the workInProgress tree. A few calls later, this function uses the DOM API to create our <h1> element, which so far has been a React element object. When the application has rendered, you can access the element by typing document.querySelector('h1'). If you check its properties, you will find one beginning with __reactInternalInstance$. This property holds a reference to the element's fiber node. React also assigns the element's text content and then later appends it to our root DOM element. By the time its appended, React has already entered its commit phase.

And that is all there is to it

In our example we have rendered the most basic of UIs but React has done a lot of work to prepare and display it. Like most frameworks created for building complex applications, it does this work to prepare for all the eventualities associated with such a task. And with the example application being static, we have not touched on any of the code involved in state updates, lifecycle hooks and the rest of it.

As this is my first time doing a deep dive on React, I am sure I have missed things out or incorrectly explained others, so please, leave your feedback below!