Forem: bandinopla

Collision detection in ThreeJs made easy using BVH

bandinopla — Mon, 25 Aug 2025 19:52:53 +0000

So you just downloaded a very cool level model that you want to test with your character and walk on it like in a game without feeling like a ghost that can pass trough geometry. Is there an easy way to do this without having to spend hours reading documentation? Yes there is!

And that's exactly what I did when I coded my "Silent Hill Demo" . I found the school level in Sketchfab and Sherry's model from Resident evil and wanted to combine the two. For the collision detection ( with the floor, walls, tables, doors, etc... ) I went the three-mesh-bvh route.

What is BHV?

I always think that real world analogies are the best way to understand things, realistic scenarios, so, with that goal in mind, imagine Ana de Armas is celebrating her birthday, and since you are dating her, she obviously invites you over, but somehow Sydney Sweeney finds her way into the party, and she is angry with Ana because she has a crush on you and can't have you since you are already taken, so she decides to get close to the birtday cake and spits on it. Now, how can you know where the spit is on the cake so you avoid eating the entire thing just to find the portion with her spit? Eating the entire cake would be ludacris, so you visually see the grid formed by the slices and you only eat the slice with Sydney Sweeney's spit on it. This optimization allows you to skip the entire cake and just focus on the important part. That's what BVH does. It slices and picks one chunk only.

Colliding by shooting rays

Tipically to get your character to "hit the floor" you would "shoot a ray" from the character's origin in the downward direction, and detect if the floor mesh ( or any other solid ) was hit, in which case you will position the character at that position, giving the illusion of a solid object.

When you shoot a ray to detect collisions, every ray cast into the scene would need to be tested against every single polygon. This brute-force approach quickly becomes impractical because the number of intersection tests grows linearly with the number of polygons.

Ok, but... What the hell is BVH?

It is a technique, called Bounding Volume Hierarchy ( BVH ), a way to sort and group faces in a spatial data structure, used in computer graphics, physics engines, and ray tracing to accelerate intersection tests between rays and complex geometry. "accelerate" as in: not wasting time.

Instead of checking every polygon in a scene individually (which would be very slow), BVH organizes geometry into a tree of nested bounding volumes ( boxes... ) that enclose groups of polygons.

In the image above, for example, sherry's raycast hit going downward will only analize the polygons under her and will ignore the objects and walls and any other area outside of that room.

The image above is from the official demo, you can easily see how this BVH mechanism subdivides a level.

Ok fine, how do I implement BVH in my game/app ?

First you have to understand that this technique requires sorting and grouping polygons that are meant to exist in a common mesh. So we will need to create a new mesh that will combine all the polygons in it to allow for this method to work. It will only be used for collision detection, it won't be visible ( unless you are debugging in which case you may chose to make this collider visible )

The setup

Install

npm install three-mesh-bvh

Create the Level Collider

Collect all static meshes

During your level/world initialization is usually where you will do this. We need to create a mesh that represents all the static objects in the scene, everything that should be considered a collidable thing. Scan your scene looking for objects that are meant to be static / collidables.

const staticMeshes: THREE.Mesh[] = []; // we collect all "static" meshes

myLevel.traverse( child =>{ 
    if (child instanceof THREE.Mesh) {
        // check if this mesh should be considered static... 
        // this is up to you then...
        if( child.name.includes("static") ) // let's say we test this...
        {
            staticMeshes.push( child ); // ok, it is a static collider...
        }
    }
})

Create the level mesh collider

Once we have our static objects we are ready to create the "collider" ( the mesh that will be used to implement BVH ) It will combine all the static meshes into one big mesh.

This is done in 4 steps:

Extract the geometries from the static meshes we collected earlier
merge them into one single geometry
prepare the geometry ( optimize that geometry for bvh to work efficiently )
Generate the BVH geometry ( the object we will use to test collisions )

import * as THREE from 'three';
import { MeshBVH, MeshBVHHelper, StaticGeometryGenerator } from "three-mesh-bvh"
import * as BufferGeometryUtils from 'three/examples/jsm/utils/BufferGeometryUtils.js';
//...

const environment = new THREE.Group();
const visualGeometries: THREE.BufferGeometry[] = [];

//
// 1) extract the geometries from all the static meshes
//
staticMeshes.forEach(mesh => {
    const geom = mesh.geometry.clone();

    //
    // Because each mesh’s geometry is defined in its local space, 
    // but when you want to merge them into a single geometry 
    // you need all their vertices in the same world space.
    //
    geom.applyMatrix4(mesh.matrixWorld);
    visualGeometries.push(geom);
});

//
// 2) Merge the geometries into one
//
const colliderGeometry = BufferGeometryUtils.mergeGeometries(visualGeometries);
const colliderMesh = new THREE.Mesh( colliderGeometry ); 

environment.add( colliderMesh );

//
// 3) Prepare the data to be generated ( optimized version )
//
const staticGenerator = new StaticGeometryGenerator(environment);

// Tell it to only keep vertex positions (no normals, uvs, etc.) since those aren’t needed for collision/raycast checks.
staticGenerator.attributes = ['position'];


//
// 4) Generate the BVH geometry
//  
const mergedGeometry = staticGenerator.generate();
mergedGeometry.boundsTree = new MeshBVH(mergedGeometry);

//
// that's it!
//
const collider = new THREE.Mesh( mergedGeometry, 

    // this material is only added as a way for you to debug. In case you want to see the mesh that was generated that will be used for testing collisions.
    new THREE.MeshBasicMaterial({ wireframe: true, opacity: .1, transparent: true })
);

things you may be wondering:

staticGenerator.generate() → produces a plain merged BufferGeometry ( just vertices/indices in world space ). At this point it’s just data, nothing “smart.”
new MeshBVH(mergedGeometry) → analyzes that geometry and builds the acceleration structure. Assigning it to mergedGeometry.boundsTree is how three-mesh-bvh hooks into Three.js’s raycasting system.

Create the player collider

Now, the player and every entity you want to move and collide in this world will need also to have a setup like this.

Usually, to detect collision, the math is done using simple shapes. For characters, a Capsule is usually used because:

Smooth Movement Over Uneven Surfaces: The rounded bottom of the capsule allows the character to slide smoothly over small bumps, cracks, and up gentle slopes without getting stuck. A box collider, by contrast, has sharp corners that would catch on these small geometric details, leading to jerky, unnatural movement.
Prevents Snagging on Corners: When moving past a corner or through a doorway, the rounded sides of a capsule allow it to slide along the wall smoothly. A box collider would often snag its corner on the wall's corner, abruptly stopping the player's movement.
Good Approximation of a Character: For most upright characters ( like people ), a capsule is a much better and more realistic approximation of their overall shape than a simple sphere or a box. It's tall and rounded, just like a person's general silhouette.

So for that reason, we first need to attatch the information of the player capsule shape to the player:

// you have to adjust this depending on your model's dimensions...  
myPlayer.userData.capsuleInfo = {
        radius: 0.5,
        segment: new THREE.Line3( 
            new THREE.Vector3(), 
            new THREE.Vector3( 0, - 1.0, 0.0 ) 
        )
    }

Colliding

We have a mesh collider and our players have each their capsule info set ( their collision shape ) To start checking for collisions we have to do a few things.

This will usually happen in your update loop ( if you have constant forces like gravity always pulling entities down ) or anytime you want to move an entity.

But let's say is that time in your code where you want to test for collisions. You will have to do this:

1) Initialization and Space Transformation

The first step is to set up the necessary variables and transform the player's collision capsule into the local space of the collider object.

// tempXXXX are top-level or global variables used to be re-used...

const capsuleInfo = myPlayer.userData.capsuleInfo; // or the current entity being tested... 

// tempMat = THREE.Matrix4()
// `collider.matrixWorld` represents the collider's position and orientation in the WORLD. By inverting it we get a matrix that can transform coordinates from world space into the collider's LOCAL space
tempMat.copy( collider.matrixWorld ).invert();

tempSegment.copy( capsuleInfo.segment );

// tempSegment = THREE.Line3()
// get the position of the capsule in the local space of the collider
tempSegment.start.applyMatrix4( myPlayer.matrixWorld ).applyMatrix4( tempMat );
tempSegment.end.applyMatrix4( myPlayer.matrixWorld ).applyMatrix4( tempMat );

2) Creating a Bounding Box

Next, an Axis-Aligned Bounding Box (AABB) is created that fully encloses the player's capsule. This box serves as a quick, initial check; if this box doesn't intersect with a part of the environment, then the more complex capsule shape can't either.

// tempBox = new THREE.Box3();
tempBox.makeEmpty();

// get the axis aligned bounding box of the capsule
tempBox.expandByPoint( tempSegment.start );
tempBox.expandByPoint( tempSegment.end );

tempBox.min.addScalar( - capsuleInfo.radius );
tempBox.max.addScalar( capsuleInfo.radius );

The box is first expanded to contain the capsule's central line segment. Then, its minimum and maximum corners are pushed outwards by the capsule's radius, ensuring the entire volume of the capsule is contained within the box.

3) The BVH Shapecast

This is the core of the collision detection process. The shapecast function from three-mesh-bvh efficiently checks the capsule's bounding box against the collider's BVH tree. It consists of 2 phases:

the "broad phase" : It performs a fast box-to-box intersection test. If the player's bounding box doesn't intersect the BVH node's box, the function returns false, and the entire branch of the geometry tree is skipped, saving immense computation.
the "narrow phase." : If intersectsBounds returns true for a leaf node, this function is called for each triangle within that node. It performs the precise, more expensive check to see if the player's capsule actually touches the triangle.

collider.geometry.boundsTree.shapecast( { 
    intersectsBounds: box => box.intersectsBox( tempBox ), //<-- tempBox is the bounding box containing our player/entity 

    intersectsTriangle: tri => { 
        // (*)
    }

} );

4) Collision Resolution Inside intersectsTriangle

When a triangle is found to be intersecting the capsule, the code must calculate how to move the player to resolve the collision.

// (*)
const triPoint = tempVector; // re-using vector: this will be filled by closestPointToSegment
const capsulePoint = tempVector2; // re-using vector: this will be filled by closestPointToSegment
// distance = shortest distance between the triangle (tri) and the capsule's central line
const distance = tri.closestPointToSegment( tempSegment, triPoint, capsulePoint );
if ( distance < capsuleInfo.radius ) {

    // depth = how far the capsule has penetrated the triangle?
    const depth = capsuleInfo.radius - distance;

    // direction = the direction the player needs to be pushed to resolve the collision
    const direction = capsulePoint.sub( triPoint ).normalize();

    //
    // push the player/entity away from the collision point...
    //
    tempSegment.start.addScaledVector( direction, depth );
    tempSegment.end.addScaledVector( direction, depth );
}

5) Applying the Correction and Updating Player State

After the shapecast has checked all potential triangles and adjusted the tempSegment, the final corrected position must be applied back to the actual player object in world space.

// tempVector & tempVector2 = new THREE.Vector3() 
const newPosition = tempVector;
newPosition.copy( tempSegment.start ).applyMatrix4( collider.matrixWorld ); // local to world...

// the total displacement from the player's original position to the new, collision-free position.
const deltaVector = tempVector2;
deltaVector.subVectors( newPosition, player.position );

// apply the push... 
player.position.add( deltaVector ); // here we are assuming player is in world's space.

6) Optional: Ground Check and Velocity Adjustment

if the player was primarily adjusted vertically we assume it's on something we should consider ground...


    //how much the character’s vertical coordinate has actually changed compared to how much it should have changed if the character were still in free-fall for the same length of time?
    // The 0.25 is just a small fudge factor so we don’t demand an exact match—floating-point noise, terrain bumps, or physics solver tolerances make a perfect 0 unrealistic.
    playerIsOnGround = deltaVector.y > Math.abs( delta * playerVelocity.y * 0.25 );

// “zero-threshold re-normalization” trick.
// the code says: “Keep the intended direction, but throw away any sub-millimetre leftovers that could cause numerical trouble.”
    const offset = Math.max( 0.0, deltaVector.length() - 1e-5 );
    deltaVector.normalize().multiplyScalar( offset );

    // adjust the player model
    player.position.add( deltaVector );

    if ( ! playerIsOnGround ) {

        // using the direction of the corrected displacement (deltaVector) as the contact-normal:
        deltaVector.normalize();

        // Whatever kept me from moving further during this frame—floor, wall, or slope—take the velocity that was trying to carry me through it and strip that component away.
        playerVelocity.addScaledVector( deltaVector, - deltaVector.dot( playerVelocity ) );

    } else {

        playerVelocity.set( 0, 0, 0 );

    }

So that's it! We corrected the position and the velocity of the player/entity to respond to the level collider.

Now go on and start using BVH!

Effortless Serverless Multiplayer in Three.js with Trystero

bandinopla — Sun, 24 Aug 2025 18:13:48 +0000

Let’s say you want to do a quick prototype in ThreeJs and play with your friends, but you quickly realize how hard mutliplayer programming is :( Did you know that the multiplayer doesn’t have to be a headache? There’s a magical tool available to add multiplayer easy and free: Trystero to the rescue!

Trystero is a minimalist, open-source JavaScript library (MIT-licensed) that empowers you to build multiplayer web apps without needing your own server infrastructure. It handles peer matching using WebRTC and various decentralized signaling backends — like BitTorrent, Nostr, MQTT, IPFS, Supabase, and Firebase — so all your app data flows directly between clients, fully encrypted end-to-end. Trystero also offers developer-friendly abstractions like rooms, event broadcasting, data chunking, progress tracking, and even React hooks — making it a smooth fit for creating serverless multiplayer experiences with Three.js

And that’s exactly what I used for my multiplayer tech demo! which I was able to code in just a day thanks to how Trystero makes it!

A real world example

Instead of explaining what trystero is, I’ll show you how I used it to make a multiplayer demo, a real use-case. You can refer to their documentation for the details.

The demo I used is here. For this demo I automatically put a new player in a queue looking for a free room, let him/her interact and log them off after a few seconds and repeat the process. This is how I did it:

Work in layers!

So, before doing multiplayer stuff, you may first want to start with single player mindset, but in a way that the player’s input is not tied to the entity being moved. This means, always code first separating the logic for input from the logic for manipulating the avatar/entity. What does it means? It means, your entity should have an interface with the methods you want to be “controllable” clearly exposed and public. Think of your players or entities as a control board with buttons. You don’t care who press the buttons, a finger, a cat or a stick, you just care about what you do once a button is pressed.

interface IEntity {
   walkTo(targetPosition: Vector3, from?: Vector3):void
   sayHi():void
   wakeup(playerId: string):void;
   // etc...
}

That way, your game entities have no idea of what input mechanic is used, they will just respond to their interface. And internally, you may use a simple state machine to handle the states, transitions, etc. The entity is input agnostic. And that’s what will make hooking trystero super easy.

The setup

At this point, we assume you have a “single player application”, and controllable entities implement some interface of your choosing. You did all the testing to make sure they work ( like for example, you can do a setTimeout and call entity.walkTo and see if it moves well and stops at destination, etc ) . Now you want to “hook” multiplayer capabilities!

So you install trystero:

npm install trystero

Players connect directly through P2P with no server in between. But before that connection can happen, they need a way to discover each other. That’s where Trystero comes in, offering multiple signaling options. I chose the Firebase strategy since it felt simpler and more reliable for a quick demo.

The room

In Trystero, everything happens inside a “room.” You can think of it like a shared stage where all the players gather , every move, action, or event takes place within that space, and everyone present instantly sees what the others are doing.

import { joinRoom } from 'trystero/firebase' // because I chose Firebase...
//...
const serverConfig = { appId: "https://XXXXXXX.firebaseio.com" }; // because I chose Firebase strategy
const room = joinRoom( serverConfig, "mySuperFunRoom");

How easy was that? “room” now is our interface to “the outside world”, it is what we will be using to “talk to the others” in the room.

Actions and Listeners

So we have a reference to a room. Now what? Well, logically, one would ask “what should I do now?” and that’s exactly what the room is expecting for you to do! Tell the room what can be done, and, if something is done, what to do in response? We do that like this:

These are the real hooks I used in the demo, a copy paste of them:

// these are extracted from my demo app to give a real life example:
const [askStatus, onSomeoneAskStatus] = this.room.makeAction<PeerId>("askstatus");
const [sendStatusOf, onStatusOf] = this.room.makeAction<PlayerStatus>("statusof"); 
const [sendPlayerMoved, onSomeoneMoved] = this.room.makeAction<MoveCommand>("move");
const [sendChat, onChat] = this.room.makeAction<string>("chat");
const [sendHi, onPlayerHi] = this.room.makeAction<null>("hi");

Actions and response to actions are handled similar to react hooks. You call “room.makeAction” and it will return a tuple where the first element is the function that will trigger the action, and the second one is a function you will use to subscribe a callback to in case of that action being triggered by someone ( trystero will execute that callback when someone else, not you, triggers that action, because you already know when you do it, obviously)

A player has entered the room!

So we have an instance of our room, we are connected, and defined what can be done and how to respond to actions… how do we know when someone enters the room?

Meet: room.onPeerJoin(…)

//
// This will be called when someone joins the room
//
this.room.onPeerJoin(peerId => {   

    // instantiate a player and put it in "limbo"
    this.game.spawnPlayer( peerId, false );

    // get data of that player.... so we call the action "askStatus"
    askStatus( peerId, peerId ); 
});

Player joined, let’s setup!

In my demo, I did 2 things in that callback. I spawn the peer’s avatar (his player mesh) and then immediately after, I call a room function “askStatus” because someone else might know the player ( think of a player disconnecting due to his cat jumping on the keyboard and turning the PC off, he will lose connection, but other players will hold information of where he was, his items, etc… so just in case, we ask everyone in the room “hey, someone knows me?”)

 onSomeoneAskStatus( async (target, asker) => {

      if( target!=selfId ) return; // only the target can give the status, since each player is it's own source of truth

      // do I know the status of this target??
      const player = this.game.getPlayer( target );

      // I used "inLimbo" flag as a way to know if  player is still not active.
      if( player && !player.inLimbo )
      { 
          sendStatusOf( this.packPlayerStatus( player ) , asker);
      } 

      // else, it means we are not ready... ignore. 
  });

So a player enters the room, I call askStatus and then on the listener for that action I check if we have info of that player, and if we do, we call the function “sendStatusOf” that will send the status of it…

//
// we got the status of a player
//
onStatusOf( (status, by)=> 
{
    let player = this.game.getPlayer( status[0] );

    if( !player ) return;

    if( player.inLimbo )
    {
        this.game.writeChat(`${status[4]} joined the room.`);
    }
    // here we bridge the trystero world with our world...
    this.game.updatePlayerStatus( status[0], {
        position: new Vector3( status[1], status[2], status[3] ),
        username: status[4]
    })
} );

The “schizophrenic” mindset

Working with Trystero often requires a kind of “schizophrenic” mindset as a developer. In the same block of code, you’ll both emit actions and immediately need to handle those same actions as if they came from another client. It’s a strange dual perspective — you’re simultaneously the sender and the receiver, pretending you’re two different people using the app. This way of thinking is essential, because in a peer-to-peer system there’s no central authority to process events for you. Every action you fire off has to be treated as though it were broadcast by someone else, ensuring your local state stays in sync with the networked state.

The bridge

When building with Trystero, you quickly realize it doesn’t magically understand your app’s world — it only moves raw data between peers. That means you, as the developer, have to write the bridge: the layer that listens to Trystero events and translates them into meaningful changes in your scene. If a packet says a player moved or fired, your bridge code is what takes that data and updates your Three.js objects accordingly. In other words, Trystero gives you the network plumbing, but you’re responsible for mapping those signals into actions that make sense inside your 3D world.

// bridge trystero's room action call to our ThreeJs scene
onPlayerHi( (ev, who)=>{

    // Trystero told us that a player is saying hi, so we look that player
    // in our scene, and trigger it's .hi() method!
    let sender = this.game.getPlayer(who);
    if( sender )
    { 
        sender.hi();
    } 
});

It’s a dance between Trystero’s calls and calling our interface. Since we initially coded our game in a way that is input agnostic, it becomes almost trivial to hook trystero into it.

Player left the room

When a player leaves a room, trystero will call room.onPeerLeave

this.room.onPeerLeave( peerId => {

      // look for the player with that ID on our ThreeJs scene...
      let player = this.game.getPlayer(peerId);
      if(!player) return;

      // show the classic text in the chat's ui
      this.game.writeChat(`${player.username} left the room.`);

      // remove the player from our scene...
      this.game.removePlayer( peerId ); 

  });

That’s where you will want to cleanup, return entities to pools, or dispose of them. Any entity representing that player should be removed.

Conclusion

Trystero makes multiplayer in the browser feel almost effortless. By handling the heavy lifting of peer-to-peer networking, it removes the need for servers, accounts, or complex backend setups — freeing you to focus entirely on your 3D experience. Combined with Three.js, it becomes an ideal toolkit for rapid prototyping: you can spin up real-time, collaborative scenes in just a few lines of code, without worrying about technical hurdles or ongoing server costs. Whether you’re experimenting with new gameplay ideas, building interactive art, or testing multiplayer mechanics, Trystero gives you the simplest path to bring your concepts to life.

Generating Texture Atlases for Optimized Assets

bandinopla — Sat, 23 Aug 2025 17:14:01 +0000

When working with real-time graphics, reducing texture swaps is a huge performance win. One way to achieve this is through texture atlases: combining many small textures into a single larger one. Instead of binding dozens of individual textures during rendering, the GPU only needs to work with one atlas, with UVs mapped to the correct tile.

This is something I did when developing my ThreeJs browser game PIP:Skull Demo and will use it as example to showcase this technique.

Pre-Build process

When developing we can work with non optimized assets and allow each model to have it’s own texture. But once we are ready for production, we may want to optimize. This involves a pre-build step ( a script you will run before running your actual build script ) that will read the original files and write new optimized ones (you may chose to replace the original glb files with the new one or just tell your build script to read the optimized ones and ignore the development ones). For this article we will focus on GLB files, a very flexible format. We’ll assume all your game/application assets are in glb format.

Reading and writing new optimized glb files

For this, we will install the following package that will provide us with many useful tools for optimization:

npm install --save @gltf-transform/core @gltf-transform/extensions @gltf-transform/functions sharp

gltf-transform is a powerful toolkit for processing and optimizing glTF files, making it ideal for a pre-build pipeline. Instead of manually tweaking assets, you can script transformations like texture compression, mesh simplification, deduplication, and pruning unused data. By integrating it into a pre-build step, the original glTF remains untouched while the script generates a new, optimized version ready for runtime. This ensures a clean workflow where source assets stay editable, and the final exported models are lightweight, faster to load, and tailored for the target platform.

I wrote a small pre-build script to automate this process (and you will have to write one for your own project too, each project is different and needs different operations) and a createAtlas function to allow me to create texture atlases for different groups of textures ( This depends on your personal organization choice and it is arbitrary ) .

The function takes a list of textures, resizes them to a defined tile size, and packs them into a single atlas according to a grid layout. Using sharp, it composites everything efficiently and can also save the result as a .png for inspection. Alongside the atlas image, it returns metadata about tile positions, making it easy to remap assets later in shaders or materials.

/**
 * Utility class to create texture atlases. You can use this to tell it to 
 * pack a bunch of textures into a single atlas.
 * 
 * @param {string} atlasName 
 * @param {Texture[]} textures 
 * @param {[number, number]} atlasSize Numbers representing how many tileSize on the width and height. Examlple: 2 = 2 times the tileSize in width.
 * @param {[number, number]} tileSize Each tile in the atlas will have this size.
 * @param {{ name:string, top:number, left:number }[]} tiles 
 * @param {boolean} savePngToDisk
 */
async function createAtlas( atlasName, textures, atlasSize, tileSize, tiles, savePngToDisk )
{ 
    const atlasTextures = tiles.map( tile=>textures.find(t=>t.getName()==tile.name )  );

    const resizedImages = await Promise.all( 

        //filer only the textures we are going to pack 
        atlasTextures

        .map(async tex => await sharp(Buffer.from(tex.getImage()))
                .resize({ width: tileSize[0], height: tileSize[1], fit: 'contain', background: { r: 0, g: 0, b: 0, alpha: 0 } })
                .toFormat('png') //<-- you may change this to webp...
                .toBuffer()
            )
    );

    // Create atlas
    const atlas = await sharp({
        create: { 
            width: atlasSize[0]*tileSize[0], 
            height: atlasSize[1]*tileSize[1], 
            channels: 4, 
            background: { r: 0, g: 0, b: 0, alpha: 0 } 
        }
    }).composite(
        resizedImages.map((img, index) => ({
            input: img,
            left: tiles[index].left * tileSize[0],
            top: tiles[index].top * tileSize[1]
        }))
    ).png().toBuffer();

    if( savePngToDisk ) // for debugging. It will be packed in the GLB otherwise
        await fs.writeFile(`./${atlasName}.png`, atlas); 

    return {
        imageData: atlas,
        tiles,
        atlasSize,
        removeUnusedTextures() {
            atlasTextures.forEach( textureToRemove => { 

                textureToRemove.detach();       // Unlink from graph
                textureToRemove.setImage(null); // Clear binary data
                textureToRemove.dispose();      // Mark for removal

            });
        }
    };
}

A nice extra is the removeUnusedTextures() helper. Once packed, the source textures can be safely detached ( and should ) and disposed of, keeping memory usage clean and ensuring only the optimized atlas is used in runtime ( so the new glb file doesn’t hold the old textures in it )

+ Add the atlas to the document

This is how we now insert the atlas texture into the GLB file:

const targetTextures = myDoc.getRoot().listTextures().filter(...);
const myAtlas = createAtlas(...);
// This adds the atlas image into the glb
const myAtlasTexture = myDocument.createTexture('MyAtlas').setImage(myAtlas.imageData).setMimeType('image/png');

By running this step during the build process, I can ship fewer files, smaller GPU state changes, and more predictable asset management — all of which translate into smoother rendering and faster load times.

But what about the UVs? Doesn’t that break the models?

Yes. Bye. The article is done...

Joking, we will update the UVs obviously! And this can be done automatically too! Check this out: The above’s utility function was used several times to pack different textures into one ( this is arbitrary, depends on your use case, you may want to group them by level, zone, etc.. it’s on you). So we have now groups of textures packed into single images. The next thing we must do is scan the glb file looking for any material that may be using a texture that we have packed, and, if so, we should update the material and UVs to point to the atlas. This is how that is done…

Scanning for materials

// search for all the materials in the glb file...
myAssetDocument.getRoot().listMaterials().forEach((material) => {
    //... code here
})

With a reference to the document of our glb file, we will loop each of it’s materials and check if that material exists in our atlas, like so:

// in this example we use the color texture, but you may want to scan for 
// the material.getMetallicRoughnessTexture() and/or 
// the material.getNormalTexture()
const colorTexture = material.getBaseColorTexture()
const name = colorTexture.getName();
const atlasUvTile = atlas.tiles.find( tile=>tile.name==name);
if( atlasUvTile ) {
  // This material must be updated!!
  material.setBaseColorTexture( myAtlasTexture )
  // and here you can call a function to scan the doc looking for   meshes using this material...
  // (*)
}

In the above’s code atlas is the object returned by our createAtlas utility function. If any of the tiles in that atlas has the same name of the current texture being scanned, then that’s our sign to start doing the replacement!

Updating the UVs

When multiple textures are combined into a single atlas, the geometry using those textures must also be updated. Each mesh originally referenced its own texture coordinates (UVs) aligned to an individual image ( 0 to 1 related to the single image they were using, now 0 to 1 would refer to the total width and height of the atlas containing a bunch of textures in it). After packing textures into a single large atlas, the UVs need to be scaled and offset so that they now correctly point to their assigned tile within the atlas.

We have the atlasUvTile which holds the necessary information to do the correct adjustments. Now we have to find all the meshes that are using this texture, like so, scanning our document for meshes:

// (*)

const w = atlas.atlasSize[0];
const h = atlas.atlasSize[1];
const tx = atlasUvTile.left;
const ty = atlasUvTile.top;

myAssetDocument.getRoot().listMeshes().forEach(mesh => {
    mesh.listPrimitives().forEach(prim => {
        if (prim.getMaterial() === material) 
        {
            const uvs = prim.getAttribute('TEXCOORD_0');
            if (uvs) 
            {
                const accessor = uvs.getArray();
                for (let i = 0; i < accessor.length; i += 2) 
                {
                    accessor[i] = accessor[i] / w + tx / w; // scale then offset...
                    accessor[i + 1] = accessor[i+1] / h + ty / h;
                }
                uvs.setArray(accessor);
            }
        }
    });
});

Once a material is confirmed to use the atlas, the code finds every mesh and primitive using that material. For each primitive, it accesses the TEXCOORD_0 attribute (the UV data). The UVs are then modified in place by applying a scale and offset: scale shrinks the UVs to fit the size of a single tile inside the larger atlas and offset shifts them to the correct position (tx, ty) inside the grid layout of the atlas.

In practice, this step ensures that every mesh in the optimized glTF correctly maps its surfaces to the right part of the atlas image. Without it, textures would appear scrambled because the original UVs would no longer match the combined atlas layout.

The result is an optimized glTF where multiple textures are merged into fewer images, reducing texture bindings at runtime while preserving the original visual fidelity of the model. This is an essential optimization step for real-time rendering, especially in games and web applications.

After all that you may want to run this to remove the now unused textures from the new final optimized glb:

myAtlas.removeUnusedTextures()

Caveat: Non-Tiled Textures Only

This UV remapping approach works correctly only for non-tiled textures. Since the UVs are scaled down and offset to fit within a single cell of the atlas, any material that relies on texture tiling or repeating patterns will break — the repetition will be clipped to the bounds of the assigned tile. For assets that depend on seamless tiling (like bricks, floors, or fabrics), this method isn’t suitable without additional handling. In those cases, it’s often better to keep the textures separate or explore more advanced atlas techniques.

That’s it!

By integrating texture atlases into your pre-build process, you not only streamline asset management but also significantly reduce the workload on the client’s computer. Fewer texture bindings mean less GPU overhead, faster load times, and smoother rendering. In the end, this optimization helps you deliver leaner, more efficient applications that provide a better experience for your users.

Raymarching: A Better Way to Explore NASA’s X-Ray Scans with ThreeJs

bandinopla — Fri, 22 Aug 2025 23:20:44 +0000

Recently I experimented with NASA’s Astromaterials 3D virtual library and did this variation of it using ThreeJs, a collection that provides access to digital scans of Apollo samples, meteorites, and Antarctic rocks. Instead of just viewing static models, I wanted to explore their inner structures in real time, not just the way a geologist might examine a thin slice under a microscope.

To achieve this, I built a real-time volumetric data visualization tool using raymarching. This technique allowed me to render NASA’s X-ray computed tomography (CT) data directly into interactive 3D volumes. Instead of pre-baked meshes, the rocks are reconstructed on the fly, voxel by voxel, giving a much richer sense of depth and material composition.

One key feature I implemented was an intensity-based filtering system. With it, you can interactively isolate layers or internal structures based on density values. For example, you can peel away “less dense” material from a meteorite and reveal inclusions or harder structures hidden inside. This makes the exploration process feel less like looking at a 3D model and more like conducting an actual virtual dissection of the sample.

The result is a tool that not only visualizes NASA’s CT data but also encourages a more intuitive and exploratory way of studying planetary materials. While my project is just a personal take on their library, I think it highlights how modern real-time rendering techniques can make complex scientific datasets more accessible and engaging.

Now let’s talk about raymarching:

What is Ray Marching?

Raymarching is a way of drawing 3D scenes on a computer without using traditional 3D models (no polygons or vertices involved)

Instead of having meshes made of triangles, you imagine shooting a ray (like a straight line of sight) from the camera into the scene. The ray moves forward step by step (“marching”) and at each step it asks: am I inside an object yet? how close am I to the nearest surface?

This is usually done with distance functions, which can tell you how far you are from a shape (like a sphere or cube). The ray keeps marching until it gets close enough to a surface, and then you know that’s where the object is.

It’s like sonar or echolocation, but in reverse: instead of sound bouncing back, you walk forward in little steps until you bump into something. That’s how the computer figures out what’s visible in that direction.

But wait, How to combine X Ray slices with Ray Marching?

The process started with a simple setup in Three.js: a 1x1x1 cube mesh with a custom shader material. The cube serves as the container for the volumetric data, while the fragment shader handles the heavy lifting of the raymarching process.

Here’s how it works:

Camera direction & contact point — Using the camera’s direction and the UV coordinates provided when the fragment shader is triggered, I calculated the local position inside the cube where the ray enters.
Mapping slices to planes — NASA provides CT scan data as image slices across three planes: XZ, XY, and YZ. By projecting the local position of the ray step onto each plane, I could determine the corresponding UV for each slice atlas.
Advancing the ray — As the ray marched through the volume, I sampled these slice images. Black areas were ignored (empty space), while grey or white pixels indicated material density. When the ray hit a non-empty voxel, it stopped — effectively reconstructing the surface or internal feature of the rock.

This method transforms stacks of 2D CT slices into a real-time volumetric visualization. Instead of static meshes, the rock’s structure emerges dynamically, with the raymarching process guided by the X-ray data itself.
The end result is an interactive way to peer inside NASA’s astromaterials, layer by layer, revealing veins, inclusions, and hidden textures that would otherwise remain locked within the dataset.

How is the X Ray data packed & read?

NASA’s provides an atlas for each face of the bounding box cube that wraps the sample rocks: XY, XZ and YZ like this:

And to read each frame in that atlas, you would write a function like this in GLSL shader code ( using a ShaderMaterial in theejs) Each frame in the atlas represents a layer. Think of it as cutting many tiny slices of a sweet apple pie…

vec3 sampleAtlas(sampler2D atlas, vec4 info, vec2 uv, float slicePercent) 
{ 
    // Ensure we're in [0, 1)
    float t = clamp(slicePercent, 0.0, 1.0);

    float totalFrames = info.w;
    float framesOnWidth = info.z;

    // Compute frame index - avoid int operations when possible
    float sliceIndexFloat = floor(t * totalFrames);
    sliceIndexFloat = min(sliceIndexFloat, totalFrames - 1.0);

    // Use mod for column calculation (GPU optimized)
    float column = mod(sliceIndexFloat, framesOnWidth);
    float row = floor(sliceIndexFloat / framesOnWidth);

    // Compute UV in atlas - vectorize operations
    vec2 frameOffset = vec2(column, row);
    vec2 sampleUV = vec2(
        info.x * (frameOffset.x + uv.x),
        1.0 - info.y * (frameOffset.y + uv.y)
    );

    return texture(atlas, sampleUV).rgb;
}

Since the base mesh is a cube, the ray position at any given time, being a vector 3, can be used to obtain the UV in each face of the cube since we intentionally kept the cube at 1x1x1 units. So the position itself is the UV on that face.

Atlas XY would have each frame belonging to the Z axis.
Atlas XZ would have each frame belonging to the Y axis.
Atlas YZ would have each frame belonging to the X axis. this way we can easily know what polor to use at any given point of the ray.

Optimizing Raymarching with Early Exit Based on Dominant Axis

While building my real-time volumetric renderer for NASA’s CT scan data, one of the biggest challenges was performance. Raymarching is expensive by nature, especially when sampling multiple 2D atlases per step (XY, XZ, YZ). To keep things interactive in the browser, I experimented with ways to cut down unnecessary texture lookups.

The Problem

At every ray step, I needed to sample three different slice atlases (XY, XZ, YZ) to reconstruct the density at that position. This was accurate, but costly , especially when rays were aligned close to one axis. In those cases, many of the extra samples contributed little to the final result.

The Optimization

The trick was to look at the dominant axis of the ray direction. If the ray is traveling almost entirely along one axis, then the majority of meaningful data will come from the corresponding atlas.

So instead of always sampling all three, I added an early exit:

Check which axis has the highest weight (the strongest contribution).
If one axis dominates (greater than 0.8 in my case), only sample that atlas.
Otherwise, fall back to sampling all three.
This way, rays aligned with X, Y, or Z take a big shortcut, while oblique angles still get the full accuracy.

The Result

By cutting down the number of texture lookups in these dominant cases, the shader runs significantly faster without noticeable visual loss. It’s a simple heuristic, but it has a big payoff: many rays in typical viewing angles align with a major axis, so this optimization kicks in often.

In other words, don’t do the work if the math tells you it won’t matter much.

vec3 sampleVolumeOptimized(vec3 uvs, vec3 weight) {
    // Find dominant axis to reduce texture samples
    float maxWeight = max(weight.x, max(weight.y, weight.z));

    // If one axis dominates heavily, only sample that texture
    if (maxWeight > 0.8) {
        if (weight.z > 0.8) {
            return sampleAtlas(xyAtlas, xyInfo, uvs.xy, uvs.z);
        } else if (weight.y > 0.8) {
            return sampleAtlas(xzAtlas, xzInfo, vec2(uvs.x, 1.0-uvs.z), uvs.y);
        } else if (weight.x > 0.8) {
            return sampleAtlas(yzAtlas, yzInfo, vec2(uvs.y, 1.0-uvs.z), uvs.x);
        }
    }

    // For oblique angles, sample all three (fallback)
    vec3 xyPixel = sampleAtlas(xyAtlas, xyInfo, uvs.xy, uvs.z);
    vec3 xzPixel = sampleAtlas(xzAtlas, xzInfo, vec2(uvs.x, 1.0-uvs.z), uvs.y);
    vec3 yzPixel = sampleAtlas(yzAtlas, yzInfo, vec2(uvs.y, 1.0-uvs.z), uvs.x);


    return xyPixel + xzPixel * weight.y + yzPixel * weight.x;
}

This optimization is just one of many small tricks that make real-time raymarching practical in the browser. By letting the math guide where to spend GPU effort , and where to cut corners, it’s possible to turn heavy scientific datasets like NASA’s CT scans into smooth, interactive visualizations. It’s a reminder that performance often comes not from doing more, but from knowing when you can safely do less.