Forem: Patrick Hund

Animating Camera Movement in Three.js

Patrick Hund — Wed, 16 Dec 2020 10:37:17 +0000

I'm building a social media network and collaboration tool based on mind maps, documenting my work in this series of blog posts. Follow me if you're interested in what I've learned along the way about building web apps with React, Tailwind CSS, Firebase, Apollo/GraphQL, three.js and TypeScript.

Today's Goal

Let's figure out how we can make a 3D model we build with three.js interactive – when we click on it, the camera moves to put the clicked object to the center of the screen, like this:

I currently need this for a project I'm working on, Nuffshell, a social network and collaboration tool based on mind maps.

For this part of my series, however, I'm not going to work on my Nuffshell code, but build something from scratch, to make it easier for you to follow along.

If you do want to follow along, I recommend you use CodeSandbox and create a new project using the “Vanilla” template:

This already has everything set up for you to work with JavaScript modules.

Step 1: Creating the 3D Scene

As a first step, I'm going to set up my basic 3D scene that shows four colored cubes, for now, without interactivity or animation.

Functions, Functions, Everywhere

I find it good practice to factor every part of my program into functions that do one thing exactly. A function should not be longer than a screen length.

With this in mind, here are the JavaScript modules I create for setting up my 3D scene. Each module contains one function.

createRenderer.js



import * as THREE from "three";

export default function createRenderer() {
  const app = document.getElementById("app");
  const renderer = new THREE.WebGLRenderer({ antialias: true });
  renderer.setSize(window.innerWidth, window.innerHeight);
  app.appendChild(renderer.domElement);
  return renderer;
}

My createRenderer sets up the 3D renderer and attaches it to the HTML page (through a DOM element called app).

I'm setting the size so that the 3D scene fills out the whole browser viewport.

createScene.js



import * as THREE from "three";

export default function createScene() {
  const scene = new THREE.Scene();
  scene.background = new THREE.Color(0xffffff);
  return scene;
}

In this function, I'm setting up a scene. Scenes in three.js are the top-level containers for all the 3D objects to be rendered.

createCamera.js



import * as THREE from "three";

export default function createCamera() {
  const camera = new THREE.PerspectiveCamera(
    75,
    window.innerWidth / window.innerHeight,
    0.1,
    1000
  );
  camera.position.z = 5;
  return camera;
}

Making 3D models in three.js is similar to making real-life movies: you need cameras and light sources to “film” it.

This function sets up the camera.

Notice how I'm setting the Z-coordinate of the camera to 5. This means that the camera is some distance away from the object it is going to “film”.

createCube.js



import * as THREE from "three";

export default function createCube({ color, x, y }) {
  const geometry = new THREE.BoxGeometry();
  const material = new THREE.MeshLambertMaterial({ color });
  const cube = new THREE.Mesh(geometry, material);
  cube.position.set(x, y, 0);

  return cube;
}

Next up is the function for the objects to be shown in my 3D scene. It creates simple 3D cubes. I can provide a color and X/Y coordinates. The Z coordinate is set to 0 for all cubes.

The 3D objects in three.js are made up of geometries and materials.

createLight.js



import * as THREE from "three";

export default function createLight() {
  const light = new THREE.PointLight(0xffffff, 1, 1000);
  light.position.set(0, 0, 10);
  return light;
}

Like in real life, we need a light source to be able to see anything in our 3D scene, so this function is for creating that.

animate.js



export default function animate(callback) {
  function loop(time) {
    callback(time);
    requestAnimationFrame(loop);
  }
  requestAnimationFrame(loop);
}

Since we want to have an animated 3D model, we need to render our model over and over again, 60 times per second, in a never-ending loop. My animate function accepts a callback argument, which is a function that gets executed as often as the browser can handle.

I use the browser's requestAnimationFrame function for this, which the standard way to do this in pretty much all browser games.

Putting It All Together

So now that we have functions to initialize renderer and scene, create a camera, a light source and some cubes and animate them, we can finally use these to create our 3D scene.

index.js



import "./styles.css";
import createCube from "./createCube";
import createLight from "./createLight";
import animate from "./animate";
import createCamera from "./createCamera";
import createRenderer from "./createRenderer";
import createScene from "./createScene";

const renderer = createRenderer();
const scene = createScene();
const camera = createCamera();

const cubes = {
  pink: createCube({ color: 0xff00ce, x: -1, y: -1 }),
  purple: createCube({ color: 0x9300fb, x: 1, y: -1 }),
  blue: createCube({ color: 0x0065d9, x: 1, y: 1 }),
  cyan: createCube({ color: 0x00d7d0, x: -1, y: 1 })
};

const light = createLight();

for (const object of Object.values(cubes)) {
  scene.add(object);
}

scene.add(light);

animate(() => {
  renderer.render(scene, camera);
});

Note how the camera, light and comes are all added to the scene.

Our animate function makes sure the scene gets rendered over and over again. For now, pretty useless, but it will become important in the next steps.

Here's the project so far:

Step 2: Adding a Click Handler

Me need to be able to click on 3D objects to tell the camera where to move to. To achieve this, I'm adding a dependency to the npm package three.interactive to my project.

This library allows us to add event listener to our 3D objects, just like with HTML DOM nodes.

At the beginning of my index.js, I'm adding an import statement to use three.interactive:



import { InteractionManager } from "three.interactive";

In addition to the renderer, scene and camera, I'm creating an interaction manager:



const interactionManager = new InteractionManager(
  renderer,
  camera,
  renderer.domElement
);

As you can see, the interaction manager needs to be able to control the renderer, camera and the canvas DOM element that the scene is rendered to.

I change the for loop that creates the cube objects and adds them to the scene to write a log statement to the console when a cube is clicked, to see if it works:



for (const [name, object] of Object.entries(cubes)) {
  object.addEventListener("click", (event) => {
    event.stopPropagation();
    console.log(`${name} cube was clicked`);
  });
  interactionManager.add(object);
  scene.add(object);
}

Notice event.stopPropagation – this is necessary so that if objects are overlapping, only the top object handles the click. Again, this works just like click handlers on DOM nodes.

The one thing that we still need to do is edit the animate loop so the the interaction manager updates with every iteration:



animate(() => {
  renderer.render(scene, camera);
  interactionManager.update();
});

Here's the project so far:

When you open up the console on this sandbox (click on “console” in the lower left), then click on the 3D cubes, you'll see the log statements issued by the click handler I've added.

Step 3: Moving the Camera

Now let's actually move the camera to the position of the cube that was clicked.

This is actually pretty straightforward – I just have to update the camera's position to match the X/Y coordinates of the cube that was clicked.

Here's the updated for loop that creates the cubes:



for (const [name, object] of Object.entries(cubes)) {
  object.addEventListener("click", (event) => {
    event.stopPropagation();
    console.log(`${name} cube was clicked`);
    const cube = event.target;
    camera.position.set(cube.position.x, cube.position.y, camera.position.z);
  });
  interactionManager.add(object);
  scene.add(object);
}

Note that while the camera's X and Y coordinates change, the Z coordinate remains the same – the camera will still have 5 units distance from the objects it is “filming”.

Here's the updated sandbox:

Click on the cubes to try it out!

Step 4: Tweening Animation

So far, when a cube is clicked, the camera jumps to the cube's position immediately. This is a step in the right direction, but actually want the camera to move over to the cube's position in a smooth motion (technically, this is called “panning”).

In short, we want to add some proper animation magic!

What is Tweening?

For creating smooth motions in animations, we use a technique called inbetweening, or “tweeting”, in short.

This technique is as old as animating itself, it was used by the artists making Snow White in the 1930s, just as it is used by the artists making animations today.

The basic idea is that you have a start and end state or something to animate (also called “keyframes”), then draw all of the states in between to create the illusion of gradual change.

Consider this bouncing ball animation:

Here, we have 3 keyframes:

Ball is on the upper left of the screen
Ball is in the bottom in the middle
Ball is on the upper right

By adding the tweens, it will look like the ball is bouncing on the floor in a smooth notion. Without them, the ball would just kind of jaggedly jump from one place to another.

JavaScript Library Tween.js

To make our camera move smoothly, we need to use tweening. Like interactivity, three.js does not provide this out of the box. We're going to have to add another npm package dependency to our project: @tweenjs/tween.js.

This library is not specifically for use with three.js. You can use it any time something should be changed over a period of time.

Let's add an import statement to index.js to use it:



import * as TWEEN from "@tweenjs/tween.js";

I create a tween in the for loop that creates my cubes, to the click handler that is fired when one of the cubes ist clicked:



for (const [name, object] of Object.entries(cubes)) {
  object.addEventListener("click", (event) => {
    event.stopPropagation();
    console.log(`${name} cube was clicked`);
    const cube = event.target;
    const coords = { x: camera.position.x, y: camera.position.y };
    new TWEEN.Tween(coords)
      .to({ x: cube.position.x, y: cube.position.y })
      .onUpdate(() =>
        camera.position.set(coords.x, coords.y, camera.position.z)
      )
      .start();
  });
  interactionManager.add(object);
  scene.add(object);
}

To add the tween, I just have to instantiate a Tween object. The argument I'm passing to the constructor is the data I want to have “tweened”. In my case, this is an object containing X and Y coordinates. At the beginning of the tween, these X/Y coords are the original camera position.

With the to method, I tell the tween library what the end state of the tweened data should be. This will be the position of the cube that was clicked.

With the onUpdate method, I determine how the data that is being tweened can be used to affect my animation. It is called for every tweening step. I use this to update the position of the camera.

Finally, the start method tells the library to start tweening right away.

One more thing – we now have to add a call to the update method of our tweening library to our animation loop:



animate((time) => {

  renderer.render(scene, camera);

  interactionManager.update();

  TWEEN.update(time);

});

The End Result

Here's the final version of our project:

When you click on a cube, the camera pans smoothly over to its position – nice!

To Be Continued…

This tutorial is part of my project diary. I'm building social media network and collaboration tool, based on mind maps. I will continue to blog about my progress in follow-up articles. Stay tuned!

Writing a TypeScript Type Definition for a JavaScript npm Package

Patrick Hund — Mon, 14 Dec 2020 12:51:01 +0000

Rant Time

My honeymoon with TypeScript is over. I've spent much more time than I would care to admit to make it work properly with my 3D mind map demo, and I'm starting to wonder if it is really worth it.

What especially threw me off was creating a type definition file (.d.ts) for npm packages that don't have types.

The official documentation on this is quite lengthy, but in my opinion fails to explain the most basic things in a plain, easy to understand way.

Unfortunately, there are surprisingly few blog articles out there on the subject, and those few are mostly outdated.

OK, got that off my chest, let's move on…

Today's Goal

I'm using this npm package in my project:

THREE.Interactive

This is a JavaScript only library, it does not provide any TypeScript type definitions, so I'll create my own.

Type Definition Files

My project is based on create-react-app. When you create a React app with --template typescript, you get everything set up for you to start using TypeScript right away.

Among other things, CRA creates a file react-app-env.d.ts in the source directory, with this content:



/// <reference types="react-scripts" />

The weird reference statement includes a bunch of pre-defined types for the project to use. For example, this makes sure that you can import styles from CSS modules in TypeScript modules without the compiler complaining.

Files ending with .d.ts are called declaration files. For adding types for the THREE.Interactive library, I can add type declarations to the react-app-env.d.ts file, or I can create an additional file next to it, for example, three.interactive.d.ts.

All these declaration files are picked up by the TypeScript compiler automatically, I don't have to configure any paths for them to be included or anything like that.

Analyzing My Code

To see which types I have to declare, I take a look at the locations in my code where I'm using the library.

Here are the relevant lines, I've left out the code that doesn't have anything to do with THREE.Interactive:

initializeScene.ts



import { InteractionManager } from "three.interactive";

const interactionManager = new InteractionManager(renderer, camera, canvas);

RenderCache.ts



import { InteractionManager } from 'three.interactive';

interface Constructor {
  interactionManager: InteractionManager;
}

export default class RenderCache {
  private interactionManager: InteractionManager;

  constructor({ interactionManager }: Constructor) {
    this.interactionManager = interactionManager;
  }

  preRender(data: MindMapData) {
    return Promise.all(
      data.nodes.map(async ({ name, val, id }) => {
        const sprite = await renderToSprite(
          <MindMapNode label={name} level={val} />
        );
        this.interactionManager.add(sprite);
      })
    );
  }
}

renderMindMap.ts



interactionManager.update();

So this means I have to add a type declaration for a class InteractionManager, which I'm instantiating in initializeScene.ts.

I have to declare two methods:

add, which I'm using in RenderCache.ts
update, which I'm using in renderMindMap.ts

The library actually does more than this, but I decide to only declare types for the stuff I'm actually using.

Adding the Declarations

I'm adding the type declarations to react-app-env.d.ts so that I end up with this:



/// <reference types="react-scripts" />

declare module "three.interactive" {
  export class InteractionManager {
    constructor(
      renderer: THREE.Renderer,
      camera: THREE.Camera,
      canvas: HTMLCanvasElement
    );

    update(): void;

    add(object: THREE.Sprite): void;
  }
}

WTF?

This works, the TypeScript compiler now makes sure I don't pass any illegal arguments to the constructor of InteractionManager or its update or add methods.

You may have noticed that my type declaration references types from the three.js library (THREE.Renderer, THREE.Camera and THREE.Sprite).

I thought I would have to import these types from three.js to make my type declaration work. Adding this to my react-app-env.d.ts seemed logical to me:



import * as THREE from 'three';

When I did this, however, the compiler gave me this error:

Could not find a declaration file for module 'three.interactive'. '/node_modules/three.interactive/build/three.interactive.js' implicitly has an 'any' type.
Try npm install @types/three.interactive if it exists or add a new declaration (.d.ts) file containing `declare module 'three.interactive'

That's right – the compiler didn't tell me that there was something wrong my my import, it just ignored the .d.ts with the import altogether.

THREE for some reason is automagically already available, I guess as a global type, in my type declaration.

Check It Out

I don't have a CodeSandbox for you to try out this article's code, because CodeSandbox seems to not work properly with TypeScript, so I ditched it after a lot of frustrating trial and error.

Here's a repository on GitHub instead:

nuffshell / nuffshell-blog-11-example

Nuffshell Blog 11 Example

Code example from the Nuffshell Coding Diary on DEV.to

View on GitHub

To Be Continued…

I'm planning to turn my mind map into a social media network and collaboration tool and will continue to blog about my progress in follow-up articles. Stay tuned!

Adding a Click Handler to a Three.js Model

Patrick Hund — Fri, 11 Dec 2020 15:59:01 +0000

Today's Goal

After spent a good part of the week procrastinating by adding Apollo State Management and TypeScript to my project, it's time to continue work on the actual mind map.

I want to start making the mind map interactive. As a first step, I want it, when a node is clicked, to zoom and pan to the clicked node, like this:

Picking a Library

Surprisingly, adding a click handler to an object in a three.js scene is not built in to the library. As was also the case with the trackball controls I've previously added, there are a couple of libraries available that add this functionality and work more or less the same.

After a bit of research, I settle for three.interactive by Markus Lerner, for these reasons:

Available as npm package
Has been updated recently
Provides ES module imports
Doesn't pollute the THREE namespace or define any global variables
Supports handling clicks on overlapping objects

The only disadvantage is that it doesn't have TypeScript types. There was one other repo on GitHub for interactive three.js models that does provide types – threejs-interactive-object. But that is not available as npm package, so I decided against using it.

Logging on Click

To see if threejs-interactive-object does what it promises, I add the npm package to my library, then set it up to log a statement to the console when a mind map node is clicked.

For this, my initializeScene function now creates and returns an InteractionManager:



function initializeScene(div: HTMLDivElement) {
  const canvas = createCanvas(window.innerWidth, window.innerHeight);
  const renderer = new THREE.WebGLRenderer({ canvas, antialias: true });
  div.appendChild(renderer.domElement);

  const scene = new THREE.Scene();
  scene.background = new THREE.Color(0xffffff);

  const camera = new THREE.PerspectiveCamera(
    50,
    window.innerWidth / window.innerHeight,
    0.1,
    500000
  );
  camera.position.z = 1.5;

  const controls = new TrackballControls(camera, renderer.domElement);

  const interactionManager = new InteractionManager(renderer, camera, canvas);

  return { scene, renderer, camera, controls, interactionManager };
}

The interaction manager is then passed to the constructor of my RenderCache, which “pre-renders” the mind map nodes, i.e. prepare them before they are displayed in the 3D model:

renderMindMap.tsx



async function renderMindMap(
  div: HTMLDivElement,
  data: MindMapData
) {
  const {
    scene,
    renderer,
    camera,
    controls,
    interactionManager
  } = initializeScene(div);
  const renderCache = new RenderCache({ interactionManager });
  await renderCache.preRender(data);
  const graph = initializeGraph(renderCache, data);
  scene.add(graph);
  camera.lookAt(graph.position);
  animate(() => {
    graph.tickFrame();
    controls.update();
    interactionManager.update();
    renderer.render(scene, camera);
  });
}

Inside the animation function call at the end, I add interactionManager.update() to make sure the interaction manager updates with every animation loop iteration.

When the mind map node is rendered, a click event handler is added that, for now, just logs a statement so we know that it works:

RenderCache.tsx



interface Constructor {
  interactionManager: typeof InteractionManager;
}

class RenderCache {
  private preRendered: Map<
    string | number | NodeObject | undefined,
    PreRendered
  > = new Map();

  private interationManager: typeof InteractionManager;

  constructor({ interactionManager }: Constructor) {
    this.interationManager = interactionManager;
  }

  preRender(data: MindMapData) {
    return Promise.all(
      data.nodes.map(async ({ name, val, id }) => {
        const sprite = await renderToSprite(
          <MindMapNode label={name} level={val} />,
          { width: 128, height: 64 }
        );
        sprite.addEventListener('click', (event) => {
          event.stopPropagation();
          return console.log(`Mind map node clicked: #${id} “${name}”`);
        });
        this.interationManager.add(sprite);
        const linkMaterial = new THREE.MeshBasicMaterial({
          color: colorsByLevel[val]
        });
        this.preRendered.set(id, { sprite, linkMaterial });
      })
    );
  }
}

Note the event.stopPropagation() – this makes sure, when mind maps nodes are overlapping, that a click only fires the event on the one closest to the camera.

Let's try it out:

👍🏻 Nice, it works!

Here's the code so far:

To Be Continued…

The next step will be to move the camera to put the mind map node that was clicked in the middle of the screen. I'll figure out how to do this in the next blog post.

I'm planning to turn my mind map into a social media network and collaboration tool and will continue to blog about my progress in follow-up articles. Stay tuned!

TypeScript All the Things!

Patrick Hund — Thu, 10 Dec 2020 10:09:16 +0000

Today's Goal

In my previous posts, I've built a 3D mind map, using React and three.js.

In the last post, I added local state management with Apollo Client. Since all the code examples I learned from were written in TypeScript, I decided to migrate my project to TypeScript, as well. Today, I'll convert all the JavaScript code for my 3D mind map that I've written so far to TypeScript.

To Type or Not to Type

TypeScript extends JavaScript by adding types, promising fewer bugs and a better developer experience.

I'm not going to lie, I've had my reservations with TypeScript. Still do, actually.

I don't like how it nudges you in the direction of object oriented programming, a programming style that in my opinion can do more harm than good if used incorrectly. Functional programming, on the other hand, which I love, can be a real pain to type properly.

I also read a lot of complaints on Twitter from developers who fiddle and fight the type system to do their bidding. There are blog posts from smart people whom I respect that say it's a hype that only got so big because it is backed by mighty Microsoft.

On the other hand, coming from Java programming, I know that types can go a long way to make programming easier and take mental load off a programmer writing code – the TypeScript compiler helps you with every keystroke, telling you what you variables can and cannot contain, what kind of arguments to pass to functions, where something might be null or undefined.

So I haven't made up my mind yet if, in the teaser picture above, I'm the boy with the trumpet or the girl holding her ears shut.

I have, however, made up my mind to use TypeScript for my 3D mind map side project, so let's get into the nitty gritty!

Getting Started

If you're using create-react-app, like I do, getting started is quite easy. CRA has TypeScript “built in”. All you have to do is change the file name extension from a JavaScript module from .js to .ts, and boom – you have a TypeScript module.

TypeScript infers the types in the TS modules, so unlike Java, you don't have to write what type it is every time you create a variable. The TS compiler will just assume type any when it can't figure out by itself what type something is.

As you can see here, when the compiler runs in strict mode, it will complain about “implicit any” type in these cases – great! My goal is to not ever use “any” anywhere. I think only then using TypeScript really makes sense.

Null Checking

One thing I've noticed while converting my code to TypeScript: the compiler warns me about something that may be null or undefined in cases where I just didn't bother with checking. I didn't bother checking because from my experience, I can rely on something to be defined/not null. The TS compiler of course can't judge from experience or gut feeling, it tries to help me and warn me.

Take the old JavaScript code of my MindMap React component, for example:

function MindMap({ data }) {
  const divRef= createRef();
  useEffect(() => {
    renderMindMap(divRef.current, data);
  }, [divRef, data]);
  return <div ref={divRef} />;
}

It's just rendering a div to the DOM, then passing a reference to the DOM node to my renderMindMap function that creates the 3D model of the mind map.

Converted to TypeScript:

interface Props {
  data: MindMapData;
}

export default function MindMap({ data }: Props) {
  const divRef: RefObject<HTMLDivElement> = createRef();
  useEffect(() => {
    renderMindMap(divRef.current, data);
  }, [divRef, data]);
  return <div ref={divRef} />;
}

I have to define an interface for the props to tell TypeScript what type of data can be passed to the component – great!

But what's this?

TypeScript thinks divRef.current could be null, so I'm not allowed to pass it to the renderMindMap function, which expects a DOM element as first argument!

I add a null check to make the compiler happy:

function MindMap({ data }: Props) {
  const divRef: RefObject<HTMLDivElement> = createRef();
  useEffect(() => {
    const div = divRef.current;
    if (!div) {
      console.error("Rendering the mind map div element failed");
      return;
    }
    renderMindMap(div, data);
  }, [divRef, data]);
  return <div ref={divRef} />;
}

I actually don't think the ref could ever be null, so did TypeScript, in this case, help me prevent a bug, or did it just force my to write extra code? 🤔 Debatable…

When I'm 100% sure I know better than the TypeScript compiler and something simply cannot be null or undefined, I can use ! to override the null check:

function MindMap({ data }: Props) {
  const divRef: RefObject<HTMLDivElement> = createRef();
  useEffect(() => {
    renderMindMap(divRef.current!, data);
  }, [divRef, data]);
  return <div ref={divRef} />;
}

❤️ Thanks Daniel for pointing this out in the comments!

Adding My Own Types

When including a library in your project, usually by installing an npm package, those libraries need to have type definitions if you want to use them properly with your TypeScript project.

Luckily, all the libraries I've included in my 3D mind map project so far do have types. It's great to see that TypeScript nowadays is already so widely supported! 👍🏻

There is only one dependency that doesn't have types, three-trackballcontrols. I'm using this to be able to zoom, pan and rotate my model (see previous post).

So, what to do?

I have to add my own type definitions. Create-react-app comes with a type definition file react-app-env.d.ts that I can use to add my type definition:

declare module 'three-trackballcontrols' {
  declare const TrackballControls: any;
  export default TrackballControls;
}

With this, I can at least import and use the library in my code without compiler warnings. Of course, it doesn't add any value, because I'm just saying the constructor for the TrackballControls object is a function that can accept any old arguments and returns who knows what.

“But Patrick, you said your goal is to not use any anywhere!” – yes, yes, I should really create a proper type definition here. Someday, somewhere, somehow… 😅

Update!

There actually is a TypeScript version of the library, three-trackballcontrols-ts.

When looking for a TS compatible version of an npm package, or for type definitions for a package that you can install separately, it is always a good idea to search on the type search page of the official TypeScript website.

❤️ Thanks stereobooster for pointing this out in the comments!

CSS Modules

Another thing I had to do some research one is using CSS modules properly. I have one CSS file per React component, containing the styles for this particular component. I can import the CSS files in the TypeScript module thanks to some dark webpack magic.

TypeScript is not happy about this:

To fix this, I add this custom type definition to my react-app-env.d.ts file:

declare module '*.css' {
  interface IClassNames {
    [className: string]: string;
  }
  const classNames: IClassNames;
  export = classNames;
}

It's a good thing that TypeScript has been around for some time now and is widely used, so in cases like this, someone else already had the same problem long ago and fixed it. I can just copy my solutions off StackOverflow or GitHub.

Enforcing Good Practices

One thing I like as I convert my project to TypeScript that the compiler calls me out in cases where I've used bad coding practices and forces me to do better.

Let me explain with one example:

renderMindMap.js (before converting)

data.nodes = await Promise.all(
  data.nodes.map((node) =>
    renderToSprite(<MindMapNode label={node.name} level={node.level} />)
  )
);
const graph = new ThreeForceGraph().graphData(data);
graph.nodeThreeObject(({ sprite }) => sprite);

This code is preparing the nodes of my 3D mind map by pre-rendering them asynchronously. The ThreeForceGraph library has a method that allows me to pass custom objects for the graph nodes, which I'm using to pass the pre-rendered sprites.

What's wrong with this code?

The object data was passed as a function argument. It contains all the data of my mind map. I'm blatantly mutating this object by adding the pre-rendered mind map nodes, for ThreeForceGraph to use. Mutating an object that is passed as an argument to a function makes this function impure and is a bad coding practice, indeed.

With TypeScript, I have to define a type for my mind map data. I tried defining the type of data.node so that it contains a sprite. However, when ThreeForceGraph passes this data to the nodeThreeObject callback function, the TypeScript compiler notices that there is a sprite property in there that, according to the type definition of ThreeForceGraph should not be there.

I fix this by creating a separate map of pre-rendered nodes and then access this map in nodeThreeObject:

renderMindMap.tsx

const preRendered: Map<
  string | number | NodeObject | undefined,
  PreRendered
> = new Map();
await Promise.all(
  data.nodes.map(({ name, val, id }) =>
    renderToSprite(<MindMapNode label={name} level={val} />)
    .then((sprite) => 
      preRendered.set(id, { sprite, linkMaterial });
    )
  )
);
const graph = new ThreeForceGraph().graphData(data);
graph.nodeThreeObject(({ id }) => {
  const sprite = preRendered.get(id)?.sprite;
  if (!sprite) {
    console.error(`Error – no pre-rendered mind map node for ID ${id}`);
    return new THREE.Mesh(
      new THREE.BoxGeometry(),
      new THREE.MeshBasicMaterial({ color: 0xffffff })
    );
  }
  return sprite;
});

Note how, again, in this case I have to add a null check, even though I'm pretty sure that, after having pre-rendered a sprite for every ID, it cannot happen that the map will return null. Oh, well…

But it is a good thing that TypeScript makes me collect the pre-rendered sprites in a separate map instead of just adding them to the original data. “TypeScript made me a better developer.” 😂

The Result

These were just a few things I've noticed while working with TypeScript. I hope you found them interesting. If you want to take a look at the whole project after conversion to TS, here's the code sandbox:

To Be Continued…

I'm planning to turn my mind map into a social media network and collaboration tool and will continue to blog about my progress in follow-up articles. Stay tuned!

Local State Management with Apollo Client

Patrick Hund — Wed, 09 Dec 2020 12:05:47 +0000

Today's Goal

In my previous posts, I've built a 3D mind map, using React and three.js.

For this project, I've used CodeSandbox exclusisvely for coding and I just love it! It works just like VS Code, but runs completely in the browser and lets me share my code easily in my blog posts.

Just one thing that bothered me as I worked on my mind map: Performance issues. The nature of rendering an interactive 3D model involves having a loop that continuously updates, running 60 times per second. CodeSandbox seems to have problems with this when my demo is running in the preview window, writing code slows down and becomes a pain.

To fix this, I'm going to add a “pause” button to my mind map demo that I can use to start/stop the rendering loop.

Picking a State Management Library

I know I'm going to have to add some way of managing application state to my React app at some point. My plan is to ultimately have a web app where users log in to collaborate on mind maps with others.

My “pause” button is the first use case that actually requires an application state, so it's time to think about state management.

I've been using Redux in my day job for five years now. Some people think it's overly complicated. I disagree, especially with the latest version of Redux Toolkit, it's become a lot easier to adopt and use.

For small projects, I also like Zustand a lot – much more light-weight than Redux.

Then Recoil popped up this year and looks really promising, especially considering it is backed by Facebook, the company behind React.

However, for my collaborative mind mapping project, I've decided to go with Apollo Client.

This is much more than just a state management library. Apollo is a framework for storing and fetching data with GraphQL, an API query language.

I will need to store my users' mind map data in a database. GraphQL is a great way to access this stored data that I've been wanting to try out for a long time now.

Toggling my 3D animation loop on and off can, of course, be achieved much, much easier with far less overhead. Sooner or later, though, it is going to pay off to have one framework for managing my local application state and remote data storage.

So let's play Rube Goldberg and crack a nut with a sledgehammer – let's toggle a single boolean value with Apollo Client!

The Code

ApolloProvider

After adding the npm packages @apollo/client and graphql to my project, the first step is to initialize the client and add an ApolloProvider component:

index.js



import React from 'react';
import ReactDOM from 'react-dom';
import { ApolloClient, ApolloProvider } from '@apollo/client';
import { cache } from './storage';
import App from './App';

const client = new ApolloClient({
  cache
});

const rootElement = document.getElementById('root');
ReactDOM.render(
  <React.StrictMode>
    <ApolloProvider client={client}>
      <App />
    </ApolloProvider>
  </React.StrictMode>,
  rootElement
);

Cache

Notice how the client in the code example above is initialized with a cache. The cache, in Apollo Client, is the central module for managing data:

cache.ts



import { InMemoryCache } from '@apollo/client';
import { appConfigVar } from './appConfig';

const cache: InMemoryCache = new InMemoryCache({
  typePolicies: {
    Query: {
      fields: {
        appConfig: {
          read() {
            return appConfigVar();
          }
        }
      }
    }
  }
});

export default cache;

Note how the cache module is written in TypeScript. So far, I've been using JavaScript for my project. The tutorial I'm following along is written in TypeScript. I was planning to convert to TypeScript at some point, anyway, so I decide to use TypeScript for the Apollo modules in my project.

Reactive Variables

The appConfigVar in my cache is a reactive variable. This is where the magic happens – the idea is to create reactive variables for everything that is stored locally, i.e. not through a GraphQL API.

appConfigVar.ts



import { makeVar } from '@apollo/client';
import AppConfig from './AppConfig';
import initialAppConfig from './initialAppConfig';

const appConfigVar = makeVar<AppConfig>(initialAppConfig);

export default appConfigVar;

AppConfig Interface

In TypeScript, we define types or interfaces to help the compiler check if everything is typed correctly.

AppConfig.ts



interface AppConfig {
  isPaused: boolean;
}

export default AppConfig;

As you can see, I'm using AppConfig objects to store the state of my application. So far, this contains only one boolean value, isPaused. The nut I'm cracking with my sledgehammer.

Initial App Config

In appConfigVar.ts, I'm using initalAppConfig to set up my reactive variable.

initialAppConfig



import AppConfig from './AppConfig';

const initialAppConfig: AppConfig = JSON.parse(
  window.localStorage.getItem('nuffshell.appConfig')
) || {
  isPaused: false
};

export default initialAppConfig;

I want my app state to be persistent, even when I reload the page in the browser. To achieve that, I'm storing it in the browser's localStorage.

When the app config reactive var is initialized, I'm checking the local storage for a previously saved app config. If there is one, I'm using this, otherwise, I use a default one, with isPaused: false.

Query to Get the App Config

To get the app config, I define a GraphQL query:

GetAppConfig.ts



import { gql } from '@apollo/client';

const GetAppConfig = gql`
  query GetAppConfig {
    appConfig @client {
      isPaused
    }
  }
`;

export default GetAppConfig;

Notice the @client part in the query definition – this tells Apollo Client that the app config comes from a local state, i.e. it does not have to be fetched through the GraphQL API.

Custom Hook

I've decided to write a custom hook to wrap up all that Apollo goodness and to be able to conveniently use it from my React components:

useAppConfig.ts



import { useQuery } from '@apollo/client';
import appConfigVar from './appConfigVar';
import GetAppConfig from './GetAppConfig';
import saveAppConfig from './saveAppConfig';

export default function useAppConfig() {
  const {
    data: { appConfig }
  } = useQuery(GetAppConfig);

  return {
    isPaused: appConfig.isPaused,
    togglePause() {
      appConfigVar({ ...appConfig, isPaused: !appConfig.isPaused });
      saveAppConfig();
    }
  };
}

As you can see, I'm using the useQuery hook from the Apollo Client to get the current value of the app config by passing my GetAppConfig query.

My hook returns an object that allows React components to get the current state (is the app paused?) and toggle the pause on/off.

Persisting the Config in Local Storage

In my custom hook, I'm calling this function saveAppConfig to store my config in the browser's local storage:



import appConfigVar from './appConfigVar';

export default function saveAppConfig() {
  window.localStorage.setItem(
    'nuffshell.appConfig',
    JSON.stringify(appConfigVar())
  );
}

Toggle Button Component

Here's the useAppConfig in action, in the PauseButton component:



import React from 'react';
import { useAppConfig } from '../../storage/appConfig';
import styles from './PauseButton.module.css';

export default function PauseButton() {
  const { isPaused, togglePause } = useAppConfig();

  return (
    <button className={styles.PauseButton} onClick={togglePause}>
      {isPaused ? 'unpause' : 'pause'}
    </button>
  );
}

Adding the Button

I'm adding this button component to my App component:



import React, { createRef, useEffect } from 'react';
import { PauseButton } from './features/pauseButton';
import renderMindMap from './renderMindMap';

export default function App() {
  const divRef = createRef();
  useEffect(() => renderMindMap(divRef.current), [divRef]);
  return (
    <>
      <PauseButton />
      <div ref={divRef} />
    </>
  );
}

Pausing the 3D Rendering

Finally, inside the loop that runs 60 times per second to re-render my 3D model, I can get the current state, asking: Is the app paused?

renderMindMap.js



(function animate() {
  const { isPaused } = appConfigVar();
  if (!isPaused) {
    graph.tickFrame();
    controls.update();
    renderer.render(scene, camera);
  }
  requestAnimationFrame(animate);
})();

In this case, I'm not using the useAppConfig hook, because this is not a React component. I can simply get the app config by calling my reactive variable appConfigVar.

The Result

Wow, what a wild ride – so much code for such a little thing to achieve! Even old-school Redux is simpler. I hope it will be worth it in the long run, when I fetch and write user data through a GraphQL API and can then handle everything through Apollo.

To Be Continued…

I'm planning to turn my mind map into a social media network and collaboration tool and will continue to blog about my progress in follow-up articles. Stay tuned!

The Nuffshell Coding Diary

Patrick Hund — Tue, 08 Dec 2020 11:16:50 +0000

I'm building a social network and collaboration tool based on mind maps, documenting my work in this series of blog posts. Follow me if you're interested in what I've learned along the way about building web apps with React, Tailwind CSS, Firebase, Apollo/GraphQL, three.js and TypeScript.

Part I: Drawing a Mind Map with Three.js and React

TL;DR: I've figured out how to render React components onto a 3D model.

Part II: Drawing a Mind Map with Three.js and React, for Real This Time

TL;DR: I've managed to arrange the child nodes around my mind map root in a circle.

Part III: Still Trying to Draw a Mind Map with Three.js and React

TL;DR: I struggled with adding the next level of mind map child nodes.

Part IV: Drawing a Mind Map with Force Directed Graphs

TL;DR: I threw away most of my former attempts and decided to use a library to render my mind map instead.

Part V: How to Fix Blurry Text on HTML Canvases on Mobile Phones

TL;DR: I found a way to make sure my mind map looks good on screens with high pixel per inch ratio.

Part VI: Resizing a Three.js Scene When the Browser Window Size Changes

TL;DR: I managed to re-render my mind map when the viewport gets resized.

Part VII: Adding Trackball Controls to a Three.js Scene with Sprites

TL;DR: You can now zoom, pan and rotate the 3D model of my mind map.

Part VIII: Local State Management with Apollo Client

TL;DR: I added a button to pause animations, using local state management provided by the Apollo Client library.

Part IX: TypeScript All the Things!

TL;DR: I converted my project to TypeScript.

Part X: Adding a Click Handler to a Three.js Model

TL;DR: I managed to log a statement to the browser console when a mind map node is clicked.

Part XI: Writing a TypeScript Type Definition for a JavaScript npm Package

TL;DR: I fought TypeScript and I won (sort of).

Part XII: Animating Camera Movement in Three.js

TL;DR: I added a smooth animation to the camera as it focuses on a 3D object when it is clicked.

Adding Trackball Controls to a Three.js Scene with Sprites

Patrick Hund — Tue, 08 Dec 2020 08:16:42 +0000

Today's Goal

In the previous parts of this series, I've built a mind map with three.js and React using a force-directed graph.

Some commenters have (justifiably) asked why I have to use a fully-fledged 3D rendering library to just draw a mind map – I might have just as well drawn it on a 2D canvas, or just used SVG or even DOM nodes to achieve the same without jumping through the hoops of texturing, calculating camera angles, rendering sprites, etc.

Well, my vision for the project is to have a fluid, animated user experience – my mind map should always be in motion, just like the thoughts in our minds never stop moving.

Today I want to add the ability to zoom, rotate and pan my mind map to take an important step in that direction and actually achieve a level of interactiveness that only a 3D model can provide.

Three.js Trackball Controls

Most of the examples on the three.js use a library called “Trackball Controls” to add zooming, rotating and panning to the demos. It is not really a library, more a plain old JavaScript program that someone wrote that gets copied around all over the place. The “official file” is found in the examples directory of the three.js source code.

This is “old school” JavaScript, using global variables without any support for module import, but luckily, someone wrapped it up in an npm package and added some helpful info on how to use it:

three-trackballcontrols

I add this library to my mind map demo and “hook it up” by adding some code to my initializeScene function:

function initializeScene(div) {
  const canvas = createCanvas(window.innerWidth, window.innerHeight);
  const renderer = new THREE.WebGLRenderer({ canvas, antialias: true });
  div.appendChild(renderer.domElement);

  const scene = new THREE.Scene();
  scene.background = new THREE.Color(0xffffff);

  const camera = new THREE.PerspectiveCamera(
    50,
    window.innerWidth / window.innerHeight,
    0.1,
    500000
  );
  camera.position.z = 1.5;

  const controls = new TrackballControls(camera, renderer.domElement);

  return { scene, renderer, camera, controls };
}

In addition to scene, renderer and camera, the function now also exports the controls object.

I'm using this in my renderMindMap function to update the camera position inside my animate loop, like so:

(function animate() {
  graph.tickFrame();
  controls.update();
  renderer.render(scene, camera);
  requestAnimationFrame(animate);
})();

As a result, I can now zoom, rotate and pan my 3D mind map – that was easy!

Try it out yourself here:

To zoom, use the mouse wheel (or two fingers trackpad swipe up/down on Macs)
To pan (move the viewport), right-click and drag the mouse
To rotate, left-click and drag the mouse

Looks Good, but Not Great!

What still bothers me at this point is that the connecting lines between the mind map nodes overlap the nodes when I rotate the mind map, making the text labels hard to read and just looking wonky overall.

The Fix

StackOverflow to the rescue:

Oct 1 '12

If you want some objects to render "on top", or "in front", one trick is to create two scenes -- the first scene is your regular scene, and the second scene contains the objects that you want to have on top.

First, set:

renderer.autoClear = false

Then create two scenes:

…

Open Full Answer

The trick is to manipulate the renderOrder property of the mind map node sprites (kinda like the Z-index in CSS), then call clearDepth on the renderer before each sprite is rendered.

Here is my updated code that renders the mind map nodes in my function renderMindMap:

data.nodes = await Promise.all(
  data.nodes.map((node) =>
    renderToSprite(<MindMapNode label={node.name} level={node.level} />, {
      width: 128,
      height: 64
    }).then((sprite) => {
      sprite.renderOrder = 999;
      sprite.onBeforeRender = (renderer) => renderer.clearDepth();
      return { ...node, sprite };
    })
  )
);

This has the desired effect – I can rotate my mind map to my heart's content and the connecting lines never overlap the mind map nodes:

Try it out:

To Be Continued…

I'm planning to turn my mind map into a social media network and collaboration tool and will continue to blog about my progress in follow-up articles. Stay tuned!

Resizing a Three.js Scene When the Browser Window Size Changes

Patrick Hund — Mon, 07 Dec 2020 11:33:59 +0000

The Problem

I've built a mind map that is rendered with three.js. Here's what I've got so far:

When I resize the browser's viewport (using the responsive preview feature of CodeSandbox, in this case), this happens:

First, I have a viewport size equivalent to a 13" MacBook Air. When I switch to various mobile device profiles, the mind map is cut off and not shown in the center of the viewport anymore.

The Solution

I'm going to add an window resize event listener that executes a function when the browser window resizes. This function then updates the HTML canvas element that the mind map is rendered on and tells three.js to adjust the settings of the 3D scene's camera.

The Code

Here's my code that adds the event listener, which I've added to my module initializeScene:

import { throttle } from 'lodash-es';
import setCanvasDimensions from './setCanvasDimensions';

const resizeUpdateInterval = 500;

window.addEventListener(
  'resize',
  throttle(
    () => {
      const width = window.innerWidth;
      const height = window.innerHeight;
      camera.aspect = width / height;
      camera.updateProjectionMatrix();
      renderer.setSize(width, height);
      setCanvasDimensions(renderer.domElement, width, height);
    },
    resizeUpdateInterval,
    { trailing: true }
  )
);

I'm using throttle from the Lodash library to prevent the resizing to occur too often – this way, the 3D scene is updated every 500 milliseconds while the browser window is being resized.

Besides setting the size of the renderer with setSize, I also update the width and height of the HTML canvas element that three.js renders to.

This is done in a separate function, setCanvasDimensions:

function setCanvasDimensions(
  canvas,
  width,
  height,
  set2dTransform = false
) {
  const ratio = window.devicePixelRatio;
  canvas.width = width * ratio;
  canvas.height = height * ratio;
  canvas.style.width = `${width}px`;
  canvas.style.height = `${height}px`;
  if (set2dTransform) {
    canvas.getContext('2d').setTransform(ratio, 0, 0, ratio, 0, 0);
  }
}

I'm using a trick here to make sure the canvas looks good on devices with a high pixel ratio, such as iPhones with Retina displays (see my previous post with details on this).

When I resize the window now, it works as one would expect:

Here's the whole project if you want to try it out:

To Be Continued…

I'm planning to turn my mind map into a social media network and collaboration tool and will continue to blog about my progress in follow-up articles. Stay tuned!

How to Fix Blurry Text on HTML Canvases on Mobile Phones

Patrick Hund — Mon, 07 Dec 2020 08:19:31 +0000

Today's episode is about one particular problem I've encountered and how I've fixed it:

The Problem

In the previous episodes, I've made a mind map using the 3D graphics library three.js (part I, part II, part III, part IV).

Here's the code of the previous version that has the problem:

Now, if you take a look at this with a regular monitor with, for example, full HD resolution (1920x1080), this will look like this:

Looks OK, however, when I look at it on my iPhone, it looks like this:

Ugh, you can hardly decipher the label texts!

The Cause

My phone has more pixels per inch (PPI) than my monitor. Apple calls this a “Retina” screen, but Android phones and tablets have this, as well.

My mind map is rendered on an HTML canvas element.

While the high PPI rate on my phone makes sure that the regular text on web pages is rendered more sharply (less pixelated), with HTML canvases, the effect is the opposite – the browser renders the content on the canvas as if the screen were regular, low PPI. Actually, even worse, it seems to additionally blur the content. It does not matter which browser, my mind map looks equally bad on Chrome and Safari.

The Solution

I've found the solution on this page:

Handling High DPI (Retina) displays in WebGL

The trick is as follows:

Detect if your canvas is being rendered on a device with high PPI
If the device has high PPI, increase the canvas size
To make sure that the bigger canvas looks to be the same size, use CSS styling to make it smaller

The Code

I've come up with this function that creates a canvas suitable to each device:



function createCanvas(width, height, set2dTransform = true) {
  const ratio = Math.ceil(window.devicePixelRatio);
  const canvas = document.createElement('canvas');
  canvas.width = width * ratio;
  canvas.height = height * ratio;
  canvas.style.width = `${width}px`;
  canvas.style.height = `${height}px`;
  if (set2dTransform) {
    canvas.getContext('2d').setTransform(ratio, 0, 0, ratio, 0, 0);
  }
  return canvas;
}

My createCanvas function accepts three arguments:

width, height: the dimensions of the canvas, the way it looks on the screen
set2dTransform: this toggles a specific line of code that uses setTransform to make sure 2D rendering on the canvas works correctly – true by default

Why the set2DTransform argument? I've discovered that the 3D graphics library three.js, which I'm using, throws an error if you set 2D transform, that's why I made it optional.

The function calls window.devicePixelRatio to determine if we have a regular or high PPI screen. On my monitor, the value is 1, on my iPhone, it's 2.

Some devices have fractional device pixel ratios, which can cause rendering issues, so I'm using Math.ceil to round up to the next integer.

We then set the width and height of the canvas by multiplying the input width and height with the device pixel ratio. Given width 800 and height 600, on my monitor, the canvas dimensions will be 800x600, on my phone, they will be 1600x1200.

We then set the width and height at which the canvas appears on the screen using style.width and style.height (in my example, this will be 800x600, regardless of device pixel ratio).

The result

Here's my Retina-compatible mind map:

Screenshot from my phone:

👍🏻 Looks great!

Acknowledgements

❤️ Thanks to Reddit user SydBal for pointing out the problem (see discussion)!

❤️ Thanks to StackOverflow user mynameisko for their post on SO that is the base for my solution!

To Be Continued…

I'm planning to turn my mind map into a social media network and collaboration tool and will continue to blog about my progress in follow-up articles. Stay tuned!

Drawing a Mind Map with Force Directed Graphs

Patrick Hund — Sat, 05 Dec 2020 16:04:15 +0000

In the previous part I, part II and part III, I've found out how to render React components on sprites in three.js and connect them with lines to make up a mind map that's nested two levels deep.

There's Got to Be a Better Way

I've been thinking about the solution I have this far. It is renders mind maps with a depth of two levels, and already it becomes clear that drawing a graph like this is not easy (it is mathematically speaking a graph).

I ultimately want my mind map to be potentially hundreds of levels deep and have thousands of nodes. To calculate the position of the nodes so that the don't overlap is a nontrivial problem.

A Facebook friend pointed me to the wikipedia article about graph drawing (thanks, Stefan!). It discusses different layout methods. This one seems to be the most appropriate for drawing a mind map:

In force-based layout systems, the graph drawing software modifies an initial vertex placement by continuously moving the vertices according to a system of forces based on physical metaphors related to systems of springs or molecular mechanics. Typically, these systems combine attractive forces between adjacent vertices with repulsive forces between all pairs of vertices, in order to seek a layout in which edge lengths are small while vertices are well-separated.

This is what it looks like (with tons and tons of nodes):

Martin Grandjean • CC BY-SA 3.0

Note quite what I'm aiming at, but I think I can make this work for me.

Using `three-forcegraph`

Dev.to user crimsonmed pointed me to an implementation of force-directed graphs for three.js: three-forcegraph – thanks Médéric!

To use three-forcegraph, I have to reformat the JSON file with my mind map data:

data.json



{
  "nodes": [
    { "id": "1", "name": "Interests", "val": 64 },
    { "id": "2", "name": "Music", "val": 32 },
    { "id": "3", "name": "Graphic Design", "val": 32 },
    { "id": "4", "name": "Coding", "val": 32 },
    { "id": "5", "name": "Piano", "val": 16 },
    { "id": "6", "name": "Guitar", "val": 16 },
    { "id": "7", "name": "Electronic", "val": 16 },
    { "id": "8", "name": "Procreate", "val": 16 },
    { "id": "9", "name": "Photoshop", "val": 16 },
    { "id": "10", "name": "Illustrator", "val": 16 },
    { "id": "11", "name": "Sketch", "val": 16 },
    { "id": "12", "name": "React", "val": 16 },
    { "id": "13", "name": "TypeScript", "val": 16 },
    { "id": "14", "name": "GraphQL", "val": 16 },
    { "id": "15", "name": "Firebase", "val": 16 },
    { "id": "16", "name": "Tailwind CSS", "val": 16 },
    { "id": "17", "name": "Computer Graphics", "val": 16 },
    { "id": "18", "name": "Ableton Live", "val": 8 },
    { "id": "19", "name": "Reason", "val": 8 },
    { "id": "20", "name": "Phaser", "val": 8 },
    { "id": "21", "name": "Three.js", "val": 8 }
  ],
  "links": [
    { "source": "1", "target": "2" },
    { "source": "1", "target": "3" },
    { "source": "1", "target": "4" },
    { "source": "2", "target": "5" },
    { "source": "2", "target": "6" },
    { "source": "2", "target": "7" },
    { "source": "3", "target": "8" },
    { "source": "3", "target": "9" },
    { "source": "3", "target": "10" },
    { "source": "3", "target": "11" },
    { "source": "4", "target": "12" },
    { "source": "4", "target": "13" },
    { "source": "4", "target": "14" },
    { "source": "4", "target": "15" },
    { "source": "4", "target": "16" },
    { "source": "4", "target": "17" },
    { "source": "7", "target": "18" },
    { "source": "7", "target": "19" },
    { "source": "17", "target": "20" },
    { "source": "17", "target": "21" }
  ]
}

While previously, the nodes were linked through a parent property, now the nodes and the links between them are defined as separate arrays.

My renderMindMap for now simply throws the data in a ThreeForceGraph object and renders it:



function renderMindMap(div) {
  const { scene, renderer, camera } = initializeScene(div);
  const graph = new ThreeForceGraph().graphData(data);
  graph.numDimensions(2);
  scene.add(graph);
  camera.lookAt(graph.position);

  (function animate() {
    graph.tickFrame();
    renderer.render(scene, camera);
    requestAnimationFrame(animate);
  })();
}

Note the line graph.numDimensions(2) – it's important, because by default, the graph will be three-dimensional, which is not suitable for a mind map and would lead to nodes that appear to overlap.

I need to render the graph in an animation loop, because that's how the library works, it starts out drawing all the nodes in the same spot, then the force of each node drives them apart from each other.

The result looks promising already – none of the nodes are overlapping:

Rendering the Mind Map Using the Force Directed Graph

Now, instead of those bubbles, I want the force directed graph to display the MindMapNode React components I had created in my earlier attempts (see part I of this series).

It took quite a bit of fiddling and trial and error, this is what I came up with:

renderMindMap.js



async function renderMindMap(div) {
  const { scene, renderer, camera } = initializeScene(div, data);
  data.nodes = await Promise.all(
    data.nodes.map((node) =>
      renderToSprite(<MindMapNode label={node.name} level={node.level} />, {
        width: 120,
        height: 60
      }).then((sprite) => ({ ...node, sprite }))
    )
  );
  const graph = new ThreeForceGraph().graphData(data);
  graph.nodeThreeObject(({ sprite }) => sprite);
  graph.linkMaterial(
    ({ level }) => new THREE.MeshBasicMaterial({ color: colorsByLevel[level] })
  );
  graph.linkPositionUpdate(updateLinkPosition);
  graph.numDimensions(2);
  graph.linkWidth(1);
  graph.scale.set(0.005, 0.005, 0.005);
  scene.add(graph);
  camera.lookAt(graph.position);

  let counter = 0;

  (function animate() {
    graph.tickFrame();
    renderer.render(scene, camera);
    requestAnimationFrame(animate);
  })();
}

Comments:

I have to render the mind map nodes, that are created through my MindMapNode React component asynchronously, hence the Promise.all call that renders all of them in one go and stores them in the data, to be pulled out again in graph.nodeThreeObject
I had to set the scale of the graph to tiny (0.005), otherwise my mind map nodes would have been very small because the whole graph is so big
To fix these scale issues, I also had to adjust my renderToSprite function so that it scales down the sprites to ⅙ size
I had to overwrite the function that calculates the position of the links with graph.linkPositionUpdate, just to set the z position to make the links appear behind the nodes; by default, they were rendered in front

So here you have it, my mind map rendered as a force directed graph:

To Be Continued…

I think next time, I'll see if I can add interactivity: it would be cool if I could add new mind map nodes and remove existing ones.

Still Trying to Draw a Mind Map with Three.js and React

Patrick Hund — Thu, 03 Dec 2020 14:36:54 +0000

In the previous part I and part II, I've found out how to render React components on sprites in three.js and connect them with lines to make up a mind map root node and the first level of child nodes, displayed around the root in a circle.

Taking It to the Next Level

Today's goal is to draw the child nodes of the level 1 mind map nodes, which I'll call level 2 nodes.

The challenge will be to calculate the X/Y coordinates of those nodes so that they fan out from their parent node while not overlapping each other.

This sketch shows what I'm trying to achieve. Root node is pink, level 1 nodes are purple and level 2 nodes are blue.

Some considerations:

while level 1 nodes are arranged in a circle, the level 2 nodes need to be arranged in semicircles, facing away from the direction of their parent node
it will probably be non-trivial to make it so that there can be any number of level 1 nodes for a parent level 2 node and prevent them from overlapping; I'll need to adjust the radius of the semicircle of the level 2 nodes depending on the number of nodes
even trickier: making sure that the semicircles of level 2 nodes don't overlap the adjourning level 2 nodes from other parent nodes

A Bit of Refactoring

Before I proceed with level 2, I do some refactoring:

renderMindMap.js

function renderMindMap(div) {
  const { scene, renderer, camera } = initializeScene(div);
  const root = data.find((node) => node.parent === undefined);
  const level1 = data.filter((node) => node.parent === root.id);
  root.x = 0;
  root.y = 0;
  root.level = 0;

  await addMindMapNode(scene, root);
  const radius = 2;
  for (let level1index = 0; level1index < level1.length; level1index++) {
    const { x, y } = calculateLevel1Coordinates({
      numberOfNodes: level1.length,
      parent: root,
      radius,
      index: level1index
    });
    const level1node = {
      ...level1[level1index],
      x,
      y,
      level: 1
    };
    await addMindMapNode(scene, level1node);
    addConnection(scene, {
      color: colors.magenta,
      parentNode: root,
      childNode: level1node
    });
  }
  renderer.render(scene, camera);
}

I've moved the calculation of the coordinates for the level 1 nodes to a new module.

calculateLevel1Coordinates.js

function calculateLevel1Coordinates({
  numberOfNodes,
  parent,
  radius,
  index
}) {
  const slice = (2 * Math.PI) / numberOfNodes;
  const angle = slice * index;
  const x = parent.x + radius * Math.cos(angle);
  const y = parent.y + radius * Math.sin(angle);
  return { x, y };
}

Improving the Level 1 Node Layout

In my last post, I complained that the layout of the level 1 nodes doesn't look pleasing to the eye because the distance between the node rectangles varies. Someone gave me the tip to rotate the level 1 nodes by 90 degrees. Currently, the circle of nodes starts to the right of the root node. I subtract 90° from the angle, so that the circle of level 1 nodes starts above the root node, and lo and behold – looks much better already!

Thanks PrudiiArca!

Adding Level 2 Nodes

Now I'm ready to add the next level of nodes. For now, I'm just copying the code from calculateLevel1Coordinates.js to calculateLevel2Coordinates.js, knowing full well that this will have to be adjusted, but let's just see how it turns out without any changes.

In renderMindMap.js, I add another for-loop to add the level 2 nodes to the scene:

for (let level1index = 0; level1index < level1.length; level1index++) {
  const { x, y } = calculateLevel1Coordinates({
    numberOfNodes: level1.length,
    parent: root,
    radius,
    index: level1index
  });
  const level1node = { ...level1[level1index], x, y, level: 1 };
  await addMindMapNode(scene, level1node);
  addConnection(scene, {
    color: colors.magenta,
    parentNode: root,
    childNode: level1node
  });
  const level2 = data.filter((node) => node.parent === level1node.id);
  for (let level2index = 0; level2index < level2.length; level2index++) {
    const { x: x2, y: y2 } = calculateLevel2Coordinates({
      numberOfNodes: level2.length,
      parent: level1node,
      radius,
      index: level2index
    });
    const level2node = { ...level2[level2index], x: x2, y: y2, level: 2 };
    await addMindMapNode(scene, level2node);
    addConnection(scene, {
      color: colors.violet,
      parentNode: level1node,
      childNode: level2node
    });
  }
}

The result looks weird, as expected:

Arranging the Level 2 Nodes in Half Circles

Being bad at math, I had to do a lot of trial and error to figure out how to arrange the level 2 nodes in a half circle, facing away from the parent node.

Here's what I came up with:

calculateLevel2Coordinates.js

function calculateLevel2Coordinates({
  numberOfNodes,
  parent,
  radius,
  index
}) {
  const slice = Math.PI / (numberOfNodes - 1);
  const angle = slice * index + parent.angle - (90 * Math.PI) / 180;
  const x = parent.x + radius * Math.cos(angle);
  const y = parent.y + radius * Math.sin(angle);
  return { x, y, angle };
}

Now the mind map graph looks good:

To Be Continued…

Stay tuned for the next episode where I will either:

turn my code for rendering the mind map into a recursive function to facilitate arbitrary nesting depth
OR throw most my work so far away and try rendering the nodes with a force graph (another useful suggestion someone made)

We'll see…

Drawing a Mind Map with Three.js and React, for Real This Time

Patrick Hund — Thu, 03 Dec 2020 07:23:41 +0000

In the previous part, I've found out how to render React components on sprites in three.js. My plan is to create a mind map. So far I've got the root node of my mind map displayed, yay!

Rendering the Nodes in Radial Arrangement

OK, so now to the part I have been dreading: figuring out how to arrange all the mind map nodes defined in my data.json file so that the are fanning out from the root node in a radial layout. Calculating the positions on the nodes will involve some trigonometry. I'm terrible at math…

I'll take baby steps. Let's render only the root node and the level 1 nodes for now. The level 1 nodes will be arranged in a circle around the root node.

Here's my code to find the root node and the level 1 nodes, then render them:

renderMindMap.js



import addMindMapNode from './addMindMapNode';
import initializeScene from './initializeScene';
import data from './data';

export default async function renderMindMap(div) {
  const { scene, renderer, camera } = initializeScene(div);
  const root = data.find((node) => node.parent === undefined);
  const level1 = data.filter((node) => node.parent === root.id);
  root.x = 0;
  root.y = 0;
  root.level = 0;

  await addMindMapNode(scene, root);
  const radius = 2;
  for (const level1node of level1) {
    level1node.level = 1;
    // TODO:
    //level1node.x = ?;
    //level1node.y = ?;
    await addMindMapNode(scene, level1node);
  }
  renderer.render(scene, camera);
}

The big fat TODO here is to calculate the x and y properties for each level 1 node.

I drew me a little picture to illustrate the problem:

Where else could I find the answer that on trusty old StackOverflow?

Mar 14 '11

Here's a solution using C#:

void DrawCirclePoints(int points, double radius, Point center)
{
    double slice = 2 * Math.PI / points;
    for (int i = 0; i < points; i++)
    {
        double angle = slice * i;
        int newX = (int)(center.X + radius * Math.Cos(angle));
        int newY = (int)(center.Y +

…

Open Full Answer

I translate the C# code from the StackOverflow post to JavaScript:



import addMindMapNode from './addMindMapNode';
import initializeScene from './initializeScene';
import data from './data';

export default async function renderMindMap(div) {
  const { scene, renderer, camera } = initializeScene(div);
  const root = data.find((node) => node.parent === undefined);
  const level1 = data.filter((node) => node.parent === root.id);
  root.x = 0;
  root.y = 0;
  root.level = 0;

  await addMindMapNode(scene, root);
  const radius = 2;
  const slice = (2 * Math.PI) / level1.length;
  for (let i = 0; i < level1.length; i++) {
    const level1node = level1[i];
    level1node.level = 1;
    const angle = slice * i;
    level1node.x = root.x + radius * Math.cos(angle);
    level1node.y = root.y + radius * Math.sin(angle);
    await addMindMapNode(scene, level1node);
  }
  renderer.render(scene, camera);
}

This works!

Making the Child Nodes Look Evenly Spaced

Here are screenshots of mind map nodes drawn with this code, with a varying number of level 1 nodes:

Although the child nodes are distributed evenly around the root node, in some cases it looks wonky, for example with 3, 7 or 9 child nodes. The problem is that the mind map nodes are rectangles. If they were squares or circles, it would look better (more even).

The red segments of the circle I have drawn here have different lengths. For my mind map nodes to look evenly distributed along the circle, these would have to have equal lengths, i.e. I have to take the width and height of the mind map nodes into account when calculating the angle for each node.

I have to admit, I'm at a loss how to calculate this, so I've posted questions on StackOverflow and StackExchange Mathematics, let's see how it goes.

If someone is reading this who can help, please let me know!

Connecting the Dots

Meanwhile, I continued my work with the connections between the root node and the level 1 node. Drawing lines with three.js is surprisingly hard.

When I naïvely used THREE.LineBasicMaterial and THREE.Line, as explained in the three.js documentation, I discovered that the lines were always 1 pixel thin, no matter what line width I set.

The problem is that WebGL doesn't support drawing lines really well. Quoting the docs:

Due to limitations of the OpenGL Core Profile with the WebGL renderer on most platforms linewidth will always be 1 regardless of the set value.

I resorted to using the library THREE.MeshLine, which seems like using a sledgehammer to crack a nut, since it is a powerful tool in its own right that can do much more amazing things than just drawing a straight, thick line.

addConnection.js



import * as THREE from 'three';
import { MeshLine, MeshLineMaterial } from 'three.meshline';

const lineWidth = 5;

export default async function addConnection(
  scene,
  { color, parentNode, childNode }
) {
  const points = new Float32Array([
    parentNode.x,
    parentNode.y,
    0,
    childNode.x,
    childNode.y,
    0
  ]);
  const line = new MeshLine();
  line.setGeometry(points);
  const material = new MeshLineMaterial({
    useMap: false,
    color,
    opacity: 1,
    resolution: new THREE.Vector2(window.innerWidth, window.innerHeight),
    sizeAttenuation: false,
    lineWidth
  });
  const mesh = new THREE.Mesh(line.geometry, material);
  scene.add(mesh);
}

My addConnection function is similar to addNode, it accepts as arguments a scene to add the connection (line) to and an object with additional arguments, in this case the two mind map nodes to connect.

Like the width and height of the mind map nodes in addNode, I've decided to declare the line width as constant for now.

My updated renderMindMap function that uses this now looks like this:



import addConnection from './addConnection';
import addMindMapNode from './addMindMapNode';
import colors from './colors';
import data from './data';
import initializeScene from './initializeScene';

export default async function renderMindMap(div) {
  const { scene, renderer, camera } = initializeScene(div);
  const root = data.find((node) => node.parent === undefined);
  const level1 = data.filter((node) => node.parent === root.id);
  root.x = 0;
  root.y = 0;
  root.level = 0;

  await addMindMapNode(scene, root);
  const radius = 2;
  const slice = (2 * Math.PI) / level1.length;
  for (let i = 0; i < level1.length; i++) {
    const level1node = level1[i];
    level1node.level = 1;
    const angle = slice * i;
    const x = root.x + radius * Math.cos(angle);
    const y = root.y + radius * Math.sin(angle);
    level1node.x = x;
    level1node.y = y;
    await addMindMapNode(scene, level1node);
    addConnection(scene, {
      color: colors.magenta,
      parentNode: root,
      childNode: level1node
    });
  }
  renderer.render(scene, camera);
}

Here's the whole project so far on CodeSandbox:

To Be Continued…

Stay tuned for my ongoing quest to render my perfect mind map!

Will he figure out a way to make the level 1 nodes spaced evenly?

Will he manage to add the level 2 and level 3 nodes without having them overlap?

All these questions and more may or may not be answered in the next episode! 😅

Forem: Patrick Hund

Animating Camera Movement in Three.js

Today's Goal

Step 1: Creating the 3D Scene

Functions, Functions, Everywhere

Putting It All Together

Step 2: Adding a Click Handler

Step 3: Moving the Camera

Step 4: Tweening Animation

What is Tweening?

JavaScript Library Tween.js

The End Result

To Be Continued…

Writing a TypeScript Type Definition for a JavaScript npm Package

Rant Time

Today's Goal

Type Definition Files

Analyzing My Code

Adding the Declarations

WTF?

Check It Out

nuffshell / nuffshell-blog-11-example

Nuffshell Blog 11 Example

To Be Continued…

Adding a Click Handler to a Three.js Model

Today's Goal

Picking a Library

Logging on Click

To Be Continued…

TypeScript All the Things!

Today's Goal

To Type or Not to Type

Getting Started

Null Checking

Adding My Own Types

Update!

CSS Modules

Enforcing Good Practices

The Result

To Be Continued…

Local State Management with Apollo Client

Today's Goal

Picking a State Management Library

Recommended Reading

The Code

ApolloProvider

Cache

Reactive Variables

AppConfig Interface

Initial App Config

Query to Get the App Config

Custom Hook

Persisting the Config in Local Storage

Toggle Button Component

Adding the Button

Pausing the 3D Rendering

The Result

To Be Continued…

The Nuffshell Coding Diary

Part I: Drawing a Mind Map with Three.js and React

Part II: Drawing a Mind Map with Three.js and React, for Real This Time

Part III: Still Trying to Draw a Mind Map with Three.js and React

Part IV: Drawing a Mind Map with Force Directed Graphs

Part V: How to Fix Blurry Text on HTML Canvases on Mobile Phones

Part VI: Resizing a Three.js Scene When the Browser Window Size Changes

Part VII: Adding Trackball Controls to a Three.js Scene with Sprites

Part VIII: Local State Management with Apollo Client

Part IX: TypeScript All the Things!

Part X: Adding a Click Handler to a Three.js Model

Part XI: Writing a TypeScript Type Definition for a JavaScript npm Package

Part XII: Animating Camera Movement in Three.js

Adding Trackball Controls to a Three.js Scene with Sprites

Today's Goal

Three.js Trackball Controls

Looks Good, but Not Great!

The Fix

To Be Continued…

Resizing a Three.js Scene When the Browser Window Size Changes

The Problem

The Solution

Using `three-forcegraph`