Forem: eerk

Getting started with TensorflowJS

eerk — Wed, 09 Jul 2025 20:15:22 +0000

Tensorflow is a machine learning library that lets you create all kinds of neural networks. TensorflowJS can run in the browser or in node. In this tutorial I want to create a simple classification network, just to get to grips with the terminology, pitfalls and basic workflow of tensorflowJS.

Neural Networks

A neural network is essentially an algorithm that uses weights and activation functions, which allow it to recognise patterns in the most complicated data. Try it out here!

Goal

Today's goal is to create a classification network that can learn to recognise dogs, cats and mice by looking at their features: size, weight, tail length, and ear size.

We start out with a demo dataset. We have 12 animals, each with features and a label.

const data = [
    [[18, 19, 5, 14], "dog"],    
    [[17, 18, 4, 13], "dog"],    
    [[19, 10, 6, 15], "dog"],    
    [[16, 17, 3, 14], "dog"],   
    [[3, 4, 8, 7], "cat"],       
    [[4, 3, 9, 6], "cat"],     
    [[3, 5, 7, 8], "cat"],       
    [[4, 4, 8, 7], "cat"], 
    [[0.5, 0.5, 2, 1], "mouse"],  
    [[0.3, 0.3, 1, 1], "mouse"], 
    [[0.7, 0.8, 3, 2], "mouse"], 
    [[0.6, 0.4, 2, 1], "mouse"]
];

The first thing to determine from this data is what shape the neural network should have.

There is always an input layer which has a cell for each feature. In this case four cells.
There is always an output layer which has a cell for each class that could possibly be predicted. Since we have 3 different animals this should have 3 cells.
In between there are hidden layers. In this simple example we put one hidden layer with 10 cells.

Project setup

In this example I set up a frontend Vite project with VanillaJS. The frontend will give us some nice visualisations.

npm create vite@latest
cd _your_project_name_
npm install @tensorflow/tfjs
npm install @tensorflow/tfjs-vis

From now on, we can use tensorflow in our JS files!

import * as tf from '@tensorflow/tfjs';
import * as tfvis from '@tensorflow/tfjs-vis';

Boilerplate

We can grab a code example for classification from the tensorflowjs website.

The following code creates a neural network with the layer architecture as mentioned above. The comments attempt to explain what is happening:

// create a classification model
const classificationModel = tf.sequential();

// add the hidden layer of 10 units
// for the first hidden layer we also have to supply the input shape, which has 4 units
classificationModel.add(tf.layers.dense({ units: 10, inputShape: [4], activation: 'relu' }));

// the final layer has 3 output units (cat, dog, mouse)
classificationModel.add(tf.layers.dense({ units: 3, activation: 'softmax' })); 

// this line creates the model with settings that work well for classification
classificationModel.compile({
    loss: 'categoricalCrossentropy',
    optimizer: 'adam',
    metrics: ['accuracy']
});

The relu argument denotes the activation functions that helps the neural network determine how the features are related to the labels. The softmax argument denotes how the final result should be shown in the 3 output cells.

In this case the 3 output cells will each have a probability from 0 to 1, and all three cells added up will always result in 1. For example, the output could be [0.8, 0.1, 0.1]

Training

Training the model can be done with one line of code. We use 50 epochs to improve the model. In one epoch, the model attempts to improve itself to make better predictions. More epochs improve the accuracy.

await classificationModel.fit(trainingData, labels, { epochs: 50 });

⚠️ The fit function expects separate trainingdata and labels though! That means we have to split our data into two arrays:

const data = [[18, 19, 5, 14], [...], [...], ...etc]
const labels = ["dog", "dog", "dog", "dog", "cat", ...etc]

Tensors

For the fit function to work, the data and label arrays need to be tensorflow tensors. This allows the calculations to run on the GPU of your machine. But this presents us with another problem: a tensor consists of numbers, but our labels are strings...

For classification tasks, the method for converting text labels into tensors that work for prediction is called one-hot-encoding. That means the labels become associated with one of the three output cells of the neural network.

If you look at the illustration above, we could say that:

dog is [1,0,0]
cat is [0,1,0]
mouse is [0,0,1]

We will now convert all our data manually so you can see what's going on:

const trainingData = tf.tensor2d([
    [8, 9, 5, 4],       // large dog
    [7, 8, 4, 3],       // medium dog
    [9, 10, 6, 5],      // big dog
    [6, 7, 3, 4],       // smaller dog
    [3, 4, 8, 7],       // cat
    [4, 3, 9, 6],       // cat
    [3, 5, 7, 8],       // cat
    [4, 4, 8, 7],       // cat
    [0.5, 0.5, 2, 1],   // mouse
    [0.3, 0.3, 1, 1],   // tiny mouse
    [0.7, 0.8, 3, 2],   // mouse
    [0.6, 0.4, 2, 1]    // mouse
]);

const labels = tf.tensor2d([
    [1, 0, 0],  // dog
    [1, 0, 0],  // dog
    [1, 0, 0],  // dog
    [1, 0, 0],  // dog
    [0, 1, 0],  // cat
    [0, 1, 0],  // cat
    [0, 1, 0],  // cat
    [0, 1, 0],  // cat
    [0, 0, 1],  // mouse
    [0, 0, 1],  // mouse
    [0, 0, 1],  // mouse
    [0, 0, 1]   // mouse
]);

Now our data should work with the fit function! Try it out!

Predicting

Once your fit function finished running (this is an asynchronous process), you can start making predictions on your model.

This means that we feed 4 numbers into 4 input cells of the network, and then we check what the values are of the 3 output cells of the network!

// Predict an animal with features 4,5,3,4 
const testData = tf.tensor2d([[4, 5, 3, 4]]); 
const prediction = classificationModel.predict(testData);

console.log('Prediction probabilities:');
prediction.print();

This will output a tensor that looks like this. These are the values of the last 3 cells in the neural network!

[[0.2227311, 0.6748303, 0.1024387],]

Since we decided that the 3 output cells stand for [dog, cat, mouse], you can read the prediction as follows:

22% chance of "dog"
67% chance of "cat"
10% chance of "mouse"

So the animal is most likely a cat 🐈! Congratulations! You have now got the bare basics of TensorFlowJS working!

Showing the actual label

It would be nicer to show the actual predicted label in the console. To do that, we first need to get the tensor data back from the GPU into a normal javascript variable. This is done using tensor.dataSync().

const predictedClass = prediction.argMax(1).dataSync()[0];

This gives the index of the higest value, so 0, 1 or 2. We use the index to get the right label:

const classNames = ['dog', 'cat', 'mouse'];
const className = classNames[predictedClass];
console.log(`I think it's a: ${className}`);

Visualising training

In our current code we have no idea how many epochs we need, or if our one hidden layer is enough to run the fit function. We can visualise the training with the Tensorflow Visor Library

await classificationModel.fit(trainingData, labels, {
    epochs: 250,
    callbacks: tfvis.show.fitCallbacks({ name: 'Training' }, ['loss'], {
        callbacks: ['onBatchEnd']
    })
});
tfvis.show.modelSummary({ name: 'Model' }, classificationModel);

This example shows the loss function during training. This number should converge towards 0.0 during training. It will never reach 0 though! Try to get it to reach between 0.2 and 0.4 for this exercise, by adjusting the amount of epochs.

Memory leaks

Creating a tensor is done on the GPU, and they have to be disposed of manually when no longer needed. This can be done using dispose and tf.tidy.

We also should organise our code a bit nicer in functions at least. The whole code now looks like this:

let classificationModel

async function createModel() {
    // create a classification model
    classificationModel = tf.sequential();
    classificationModel.add(tf.layers.dense({ units: 10, inputShape: [4], activation: 'relu' }));
    classificationModel.add(tf.layers.dense({ units: 3, activation: 'softmax' }));
    classificationModel.compile({
        loss: 'categoricalCrossentropy',
        optimizer: 'adam',
        metrics: ['accuracy']
    });

    // train the model
    await classificationModel.fit(trainingData, labels, {
        epochs: 250,
        callbacks: tfvis.show.fitCallbacks({ name: 'Training' }, ['loss'], {
            callbacks: ['onBatchEnd']
        })
    });
    tfvis.show.modelSummary({ name: 'Model' }, classificationModel);

    // clear the training data tensors from memory
    trainingData.dispose();
    labels.dispose();
}

// predict an animal. use tf.tidy to remove the temporary tensors right away
function predict(animal) {
    return tf.tidy(() => {
        const testData = tf.tensor2d([animal]);
        const prediction = classificationModel.predict(testData);

        console.log('Prediction probabilities:');
        prediction.print();

        // example prediction  [[0.9999997, 3e-7, 2e-8]]
        // This means: 99.99997% dog, 0.00003% chance cat, 0.000002% chance mouse

        // Get the predicted class
        const predictedClass = prediction.argMax(1).dataSync()[0];
        const classNames = ['dog', 'cat', 'mouse'];
        return classNames[predictedClass];
    });
}

await createModel()

let result = predict([5, 6, 7, 6])
console.log(`I think it's a ${result}`);

🤩 That's it! The code is still pretty concise if you consider everything that's happening under the hood!

Further improvements

Data should be normalized before training. If we don't do that, the model will incorrectly assume that a feature with a bigger range (such as the weight of the animal) is more important than a feature with lower range (such as the length of the whiskers). By normalizing the data, all features become equally important.
We should create a function that converts the training data automatically into the right format for the fit function
We should create test data and an evaluate function to test if our model actually predicts well on unknown, but labeled, animal data.
We should be able to save a model after training

🐈 This should be covered in a follow-up tutorial!

Creating a recommender system in 10 lines of javascript

eerk — Thu, 13 Apr 2023 15:45:06 +0000

I've been dabbling in Machine Learning algorithms for Javascript, coming from a background of front-end development.

AI and Machine Learning can get complex very quickly, but I found that some basic algorithms, such as a decision tree or nearest neighbour, are easy to grasp and can still be surprisingly powerful.

In this super short tutorial we use K-Nearest-Neighbour to find people who have similar interests to you.

KNN is basically the pythagorean theorem. It compares a point in a space to several other points and tells you which is the closest.

Check this codepen to see an example

The "K" in KNN stands for the amount of points you want to check. If we use a K of 1 we just get the point closest to us.

Using the X,Y axes as data

The trick of using KNN is that we can use the X,Y axes to project data. If we want to plot cats and dogs, we could draw their body weight on the X axis, and their ear length on the Y axis.

If we draw an unknown animal in this same plot, it should be easy to see if it's a cat or a dog, just by checking which are the closest animals.

Multidimensional madness

From the above examples you might get the impression that good old pythagoras was just a two-dimensional character. But fascinatingly, his formula doesn't care how many dimensions there are.

We can add a third dimension and still draw it in a graph. But we could add many more, and the math still works.

In the example of pets, we could add many more pet features to find out what kind of pet we're dealing with.

In pseudo code it could look like this:

let pet = {ears:2, weight:92, height:14, speed:93}
let prediction = knn.predict(pet)
// returns: "Jaguar"

KNN gives us the ability to classify data by pretending all the features are distances

How about those 10 lines of javascript?

Let's use this algorithm to create a recommendation system. We'll find a person that has similar interests to ourselves.

We will use this 9-years-old github repo which still seems to work! Pythagoras himself is a lot older so I don't see any problem here.

npm install knear

We are going to teach the algorithm that every point is a unique person, and all the features of that person are their interests. Then, by finding the one closest person, you can find your match.

import knn from 'knear'

const data = [
    { cooking: 1, painting: 10, name: 'erik' },
    { cooking: 0, painting: 1, name: 'bob' },
    { cooking: 10, painting: 1, name: 'ellen' },
    { cooking: 4, painting: 6, name: 'jill' },
    { cooking: 3, painting: 8, name: 'ramon' },
]

const machine = new knn.kNear(1)
for (let d of data) {
    machine.learn([d.cooking, d.painting], d.name)
}

You can find your match:

const prediction = machine.classify([1,9])
console.log(`Your closest match is ${prediction}`)

In this example we are only interested in cooking and painting (in any order), but you can add as many features as you want.

Thanks for reading!

P.S. Doesn't it boggle your mind that they already had a machine learning algorithm in 500BC???

Adding physics to web components

eerk — Fri, 28 May 2021 17:20:48 +0000

It's Friday afternoon, so I wanted to do some crazy experiment. In a previous post I already looked into using Web Components (Custom Elements) for browser game development.

Today we're going to add physics to our HTML tags, just because it's possible! And to learn a bit about web components and Matter.JS

We'll be looking at:

Custom Elements
Game Loop
Adding Physics (with Matter.js)
Project setup (with Parcel.js)

A simulation with bouncy barrels, stubborn crates, platforms and a player character.

Example code is in Typescript, but you can leave out type annotations such as a:number and public, private to convert to Javascript.

Custom Elements

A custom element is a HTML tag that has executable code added to it. That's really handy for game objects! We'll use that to add physics later. You can nest custom elements within each other to create a hierarchy. The tag names have to end with -component (at least I get an error if I leave that out)...

HTML

<game-component>
    <platform-component></platform-component>
    <crate-component></crate-component>
    <player-component></player-component>
</game-component>

CSS

We will use translate to position our elements with javascript, so that means all elements need position:absolute and display:block. You can use a background image for the visual, it's shorter and faster than using <img> tags, and you can use repeating backgrounds.

platform-component {
   position:absolute;
   display:block;
   background-image:url(./images/platform.png);
   width:400px; 
   height:20px;
}

TYPESCRIPT

First we have to bind our code to the HTML tag by creating a class and registering it using customElments.define().

😬 In Javascript this is exactly the same, except for the :number type annotations

export class Crate extends HTMLElement {
    constructor(x:number, y:number) {
        super()
        console.log(`I am a crate at ${x}, ${y}`)
    }
}

customElements.define('crate-component', Crate)

You can add it to the DOM by placing the tag in the HTML document: <crate-component></crate-component>. But if we do it by code we can pass constructor arguments, in this case an x and y position. This is handy if we want several crates at different positions:

let c = new Crate(200,20)
document.body.appendChild(c)

GAME LOOP

To use physics, we need a game loop. This will update the physics engine 60 times per second. The game loop will then update all the custom elements. In this example, we create a game class with a game loop that updates all crates.

import { Crate } from "./crate"

export class Game extends HTMLElement {
    private crates : Crate[] = []
    constructor() {
        super()
        this.elements.push(new Crate(270, 20))
        this.gameLoop()
    }
    private gameLoop(){
        for (let c of this.crates){
            c.update()
        }
        requestAnimationFrame(() => this.gameLoop())
    }
}
customElements.define('game-component', Game)

The crate component gets an update function to translate its position.

export class Crate extends HTMLElement {
    constructor(private x:number, private y:number) {
        super()
    }
    public update() {
        this.style.transform = `translate(${this.x}px, ${this.y}px)`
    }
}
customElements.define('crate-component', Crate)

🔥 PHYSICS

FINALLY we get to the point where we add Matter.js physics! Matter.js creates a physics engine that can run invisibly in the background. If we add objects such as boxes, cylinders, floors and ceilings to it, it will create a physics simulation with those objects. Our elements will respond to gravity, friction, velocity, force, bounciness and get precise collision detection.

Matter.js has a renderer that can draw those objects directly in a canvas, but that's boring 🥱. We'll use the positions of the physics elements to position DOM elements!

Plan:

1 - Adding the physics world to the game class
2 - Adding physics to the crates
3 - What more can you do with physics?

1 - Adding Matter.js to the Game class

import Matter from 'matter-js'
import { Crate } from "./crate"

export class Game extends HTMLElement {
    private engine : Matter.Engine
    private world : Matter.World
    private crates : Crate[] = []

    constructor() {
        super()
        this.engine = Matter.Engine.create()
        this.world = this.engine.world
        this.crates.push(
            new Crate(this.world, 270, 20, 60, 60),
            new Crate(this.world, 320, 70, 60, 60)
        )
        this.gameLoop()
    }
    private gameLoop(){
        Matter.Engine.update(this.engine, 1000 / 60)
        for (let c of this.crates){
            c.update()
        }
        requestAnimationFrame(() => this.gameLoop())
    }
} 
customElements.define('game-component', Game)

2 - Adding physics to the crates

The Crate class will add a physics box to the physics world. Then, it will read the physics box position in the update function, and update the crate element position in the DOM world.

import Matter from 'matter-js'

export class Crate extends HTMLElement {
    private physicsBox: Matter.Body

    constructor(x: number, y: number, private width: number, private height: number) {
        super()
        this.physicsBox = Matter.Bodies.rectangle(x, y, this.width, this.height, options)
        Matter.Composite.add(game.getWorld(), this.physicsBox)
        document.body.appendChild(this)
    }
    public update() {
        let pos = this.physicsBox.position
        let angle = this.physicsBox.angle
        let degrees = angle * (180 / Math.PI)
        this.style.transform = `translate(${pos.x - (this.width/2)}px, ${pos.y-(this.height/2)}px) rotate(${degrees}deg)`
    }
}
customElements.define('crate-component', Crate)

3 - What more can you do with physics?

We're really just getting started using Matter.JS. To build the game you see in the images from this post you use the following concepts:

Static elements

These are elements such as platforms and walls, that do not have forces applied to them, but still cause collisions.

this.physicsBox = Matter.Bodies.rectangle(x, y, w, h, {isStatic:true})

Velocity

By setting the velocity of an object manually, you can create a player or enemy character that moves according to player input.

Matter.Body.setVelocity(this.physicsBox, { x: 5, y: this.physicsBox.velocity.y })

Force

By adding force you can temporarily boost an object in a certain direction, for example a rocket or a bullet. You can use force to make a character jump.

Matter.Body.applyForce(this.physicsBox, { x: this.physicsBox.position.x, y: this.physicsBox.position.y }, { x: 0, y: -0.15 })

Project setup

You can set up the above project (with or without Typescript) using Parcel to bundle your modules:

npm install -g parcel-bundler
npm install matter-js
npm install @types/matter-js
npm install typescript

Then, you can run the project in watch mode using

parcel dev/index.html

Or build the whole project using

parcel build dev/index.html --public-url ./

Conclusion

I hope this post didn't become too long! I think this approach is great fun, but is it really useful compared to using a canvas for physics simulations? Well...

Canvas elements can't have Event Listeners
Canvas doesn't have a nice DOM tree that you can traverse

Disadvantages:

Rendering and game structure are a bit too intertwined (you can't easily switch to canvas rendering at a late stage in development).
If you want thousands (or tens of thousands) of objects bouncing around, a canvas is much more efficient.

Inventing your own HTML Elements to build a DOM game

eerk — Wed, 19 Jul 2017 14:58:20 +0000

Building a DOM game with HTML Custom Elements

This is a small experiment showing how to invent your own HTML Elements to build a game in the DOM.

What are these custom elements?

A custom element is an HTML Element that allows you to add your own properties and methods. For example, the basic HTMLElement has a style property and a click() method. By extending the HTML Element we get all this existing functionality and we can add our own.

In this experiment we have an element Car that has an x and y property and an update() method:

class Car extends HTMLElement {

    public x: number;
    public y: number;

    constructor(){
        super();
        console.log("A car was created!");
    }

    public update(): void {
        console.log("The car's update method was called!");
    }
}

Before we can add our Car element to the DOM, we have to register it, connecting our class to a HTML tag. Note that the html tag needs to contain a hyphen:

window.customElements.define("car-component", Car);

Now you can add cars to the dom by placing tags:

<body>
    <car-component></car-component>
</body>

In our game we prefer to add cars by code. We can create a new instance of Car and add it to the DOM in one line:

document.body.appendChild(new Car());

This will result in a <car-component></car-component> being added to your HTML structure, and the message A car was created! will appear in the console. This HTML Element will have an x and y property and an update() method.

DOM manipulation

You can query your HTML document for car components, and use the new for of loop to call their update() method.

let cars : NodeListOf<Car> = document.getElementsByTagName("car-component") as NodeListOf<Car>;

for(let c of cars){
    c.update();
}

Lifecycle

A custom element has lifecycle hooks: these methods get called automatically when the Car is added to, or removed from, the DOM.

class Car extends HTMLElement {

    public connectedCallback(): void {
        console.log("A car was added to the DOM");
    }

    public disconnectedCallback():void{
        console.log("hey! someone removed me from the DOM!");
    }
}

Game Loop

Our game class will create a player element and start the game loop. A game loop updates our game elements 60 times per second using requestAnimationFrame.

The game loop will add a new Car() to the DOM every second by using the modulo operator.
The game loop will find all <car-component> tags and call their update method.

The game is instantiated with new Game() after window.load. Note that the Game class itself doesn't need to extend HTMLElement.

class Game {

    private counter:number = 0;

    constructor() {
        document.body.appendChild(new Player());
        requestAnimationFrame(() => this.gameLoop());
    }

    private gameLoop(){
        this.counter++;
        if(this.counter%60 == 0) {
            document.body.appendChild(new Car());
        }

        let cars : NodeListOf<Car> = document.getElementsByTagName("car-component") as NodeListOf<Car>;

        for(let c of cars){
            c.update();
        } 

        requestAnimationFrame(() => this.gameLoop());
    }
}

window.addEventListener("load", function() {
    new Game();
});

Removing cars

When a car leaves the screen or hits the player, we can easily remove it. Since the car extends from HTMLElement we can use the remove() method, which removes it from the DOM. The game loop won't call the update method anymore, and if we want, we could use the disconnectedCallback() to execute some final code before the car is completely removed.

public disconnectedCallback():void{
    console.log("the car is removed from the game!");
}

public update(): void {
    this.x += this.speed;

    if (this.x > window.innerWidth) {
        this.remove();
    }
}

Styling and animation

Note that the styling of our custom elements is entirely done in CSS. First we declare that ALL document elements are going to use position:absolute, and then we declare a size and a background image for each individual element:

body * {
    position: absolute;
    display: block;
    margin:0px; padding:0px;
    box-sizing: border-box;
    background-repeat: no-repeat;
}

car-component {
    width:168px; height:108px;
    background-image: url('../images/car.png');
}

We position our elements using css transform, so that we can use the GPU for smooth animation.

this.style.transform = `translate(${this.x}px, ${this.y}px)`;

Extending other elements

This example only uses extends HTMLElement, but we could also extend a HTMLDivElement, HTMLButtonElement, etc. In this example, all our elements are treated as <div> by simply setting display:block in the CSS.

Typescript

This experiment is built with Typescript, but you can easily rebuild it in pure Javascript by removing the type information. You can check the main.js file to see the Javascript equivalent.

To compile this project you can install Typescript with npm install -g typescript, and then type tsc -p in the terminal in the project folder.

Browser support

The above experiment works in Safari and Chrome. Extending specific elements such as HTMLButtonElement doesn't work in any browser yet. Use the polyfill to get support in all browsers.

For of loop in NodeList

The for of loop is not yet supported for NodeList and HTMLCollection in Safari and Firefox, because they are technically not arrays. Enable it with:

NodeList.prototype[Symbol.iterator] = Array.prototype[Symbol.iterator];
HTMLCollection.prototype[Symbol.iterator] = Array.prototype[Symbol.iterator];

Download the working project at:

https://github.com/KokoDoko/game-custom-elements

Hi, I'm eerk

eerk — Wed, 19 Jul 2017 14:51:42 +0000

I teach creative technology in Rotterdam. I organise workshops about Creative Javascript, Machine Learning, React and Arduino.

I'm currently 3D printing a little BMO with a tiny tiny LED matrix and an even tinier microcontroller.

I'm super interested in learning more about Machine Learning with Javascript, and getting better at React.

Twitter : @eerk

Forem: eerk

Getting started with TensorflowJS

Neural Networks

Goal

Project setup

Boilerplate

Training

Tensors

Predicting

Showing the actual label

Visualising training

Memory leaks

Further improvements

Creating a recommender system in 10 lines of javascript

Using the X,Y axes as data

Multidimensional madness

How about those 10 lines of javascript?

Adding physics to web components

Custom Elements

HTML

CSS

TYPESCRIPT

GAME LOOP

🔥 PHYSICS

Plan:

1 - Adding Matter.js to the Game class

2 - Adding physics to the crates

3 - What more can you do with physics?

Static elements

Velocity

Force

Project setup

Conclusion

Links

Inventing your own HTML Elements to build a DOM game

Building a DOM game with HTML Custom Elements

What are these custom elements?

DOM manipulation

Lifecycle

Game Loop

Removing cars

Styling and animation

Extending other elements

Typescript

Browser support

For of loop in NodeList

Hi, I'm eerk