Forem: Talin

TypeScript "strictNullChecks" - a migration guide

Talin — Mon, 31 May 2021 05:14:38 +0000

So you want to write high quality code, right? After all, that's presumably why you chose TypeScript in the first place.

TypeScript has a number of compiler options, settable in tsconfig.json, which determine how strict or lenient the compiler is. One of the most important options is strictNullChecks.

Effect of setting `strictNullChecks`

The default setting is strictNullChecks=false. With strict null checks disabled, every value, every parameter and every object property is both nullable and optional. If you try to assign a value of null or undefined to that variable, the compiler won't complain.

let a: int = 1;
a = null; // OK

With strictNullChecks=true this is no longer allowed - attempting to assign null to a non-nullable field will produce a compiler error.

let a: int = 1;
a = null; // Error!

That seems pretty straightforward, right? Well, not so fast. You see, the compiler will also generate an error if you try to assign a value that might be null to a non-nullable variable:

let a: int = 1;
let b?: int; // Value is not yet set, so undefined
a = b; // Error!

In addition, the compiler will no longer allow you to leave a non-optional variable uninitialized:

let a: int; // Error, a must be initialized!

Benefits of `strictNullChecks`

Why is strictNullChecks important? There are a number of reasons, I will name just two:

strictNullChecks can, and often will, locate actual bugs in your code. Lots of bugs, in fact. In other words, this is not just strictness for the sake of strictness; enabling this option can be a valuable complement to writing unit tests.
With this option enabled, autocompletion and type checking in Visual Studio Code and other editors becomes considerably smarter.

That last point deserves some explanation. Enabling this option effectively narrows the set of possible types for every variable, which means that the editor can infer more about the variable's type than it would otherwise.

Migrating existing code bases

If you are starting a new TypeScript project, enabling strictNullChecks is pretty easy - and highly recommended.

However, what if you have a large code base, that was not written with strictNullChecks in mind? Enabling the strictNullChecks option is going to produce lots and lots of errors, all of which will need to be fixed.

I recently migrated a project with roughly 20K lines of source; I ended up having to fix over 600 type errors as a result.

Worse, there's no one-size-fits-all fix for these errors. Although the fixes are usually pretty simple, it can't be done with a search and replace. You have to look at each and every error and think about what is the correct thing.

Nor is there a way to conditionally enable strictNullChecks for some source files and not others. It's either all or nothing for your whole project.

Fortunately, there are some methods you can use to soften the blow, which I will outline below.

Migrating Incrementally

Unless you plan on sitting down and fixing every error in a single sitting, you're going to want a way to be able to fix a subset of the errors and then check in you work - and you won't want to break your build in doing so.

The way around this is pretty simple:

Set strictNullChecks to true.
Do a test compile. You'll see lots of errors.
Fix some of the errors; repeat until you get tired. (Pro tip: don't code when you're tired, you're more likely to make mistakes.)
Set strictNullChecks back to false.
Check in the code.
Once you've recovered, go back to step 1.

This works because any code that is correct with strictNullChecks enabled is also acceptable with it disabled; in other words, turning it off will never generate any additional errors. So even if you can't do everything in one go, you can make partial progress towards your goal. Eventually once all the errors have been fixed, you can leave the option enabled permanently.

Techniques for handling null & undefined values.

OK, so this sounds like a lot of work, right? Well the good news is that fixing null/undefined errors is generally pretty easy. Those 600 errors I mentioned? Took less than a day to get all of them.

The following sections provide some suggestions on how to handle null/undefined values. Note that goal here is not simply to make the compiler shut up - the goal of each of these techniques is to make the code more correct.

Explicit test for truthiness

TypeScript’s type inference can narrow the type of an expression inside an if-block:

const userProfile: UserProfile | null = getProfile(id);

if (userProfile) {
  // The compiler knows that the type of 'userProfile' in
  // this block is non-null, so this dereference won't be
  // flagged as an error
  console.log(userProfile.createdAt);
}

Optional chaining operator

Another way to test a value for null or undefined is to use
the optional chaining operator, ?.:

const userProfile: UserProfile | null = getProfile(id);
console.log(userProfile?.createdAt);

In this case, if userProfile is null, then it will return null from the expression instead of attempting to dereference the value.

Explicitly telling the compiler a value is non-null

An exclamation point (!) at the end of an expression tells the compiler “Hey, I know this is non-null and non-undefined, you don’t need to warn me about it.”

const userProfile: UserProfile | null = getProfile(id);
console.log(userProfile!.createdAt);

Since this is effectively disabling the null/undefined check, it should be used with caution - but there are rare cases where this is the clearest and most correct approach.

One example is where a class has a property which is not initialized in the constructor, but is initialized immediately afterwards, possibly as a result of an ‘init()’ method. Since the value is not set in the constructor it has to be marked as optional - but the programmer knows that it’s never going to be undefined afterwards, so forcing them to add an extra null check every time the property is accessed creates clutter.

Throw an exception if a value is non-null

This is a variant of the if-block technique which inverts the logic:

const userProfile: UserProfile | null = getProfile(id);
if (!userProfile) {
  throw new Error('Profile should not be null/undefined!');
}
// Type is narrowed from this point on.
console.log(userProfile.createdAt);

Note that if the value was null, the compiler would have thrown an error anyway; the advantage here is that the exception messages explains exactly what the problem was instead of requiring the programmer to figure it out.

Throw an exception II - assertDefined helper

If an if-statement with an exception is too verbose for your taste, you can define a helper function to do it for you:

const userProfile: UserProfile | null = getProfile(id);
assertDefined(userProfile, 'userProfile in function X');
// Compiler knows that userProfile is non-null/undefined.
console.log(userProfile.createdAt);

How does the compiler know that the value of userProfile is defined after the call to assertDefined? The magic is in the return type of the helper function, which uses the TypeScript asserts feature:

/** Helper function which throws an exception if the value
    is either null or undefined. This allows TypeScript to
    correctly infer the non-nullable type.
*/
export function assertDefined<T>(value: T, name?: string): asserts value {
  if (value === null) {
    throw new Error(`${name ?? 'Value'} should not be null.`);
  }
  if (value === undefined) {
    throw new Error(`${name ?? 'Value'} should not be undefined.`);
  }
}

The second argument is optional, but recommended - it can help you track down where in the code the null value was. The error message will be something like “userProfile in function X should not be null.”.

Use getter methods to tighten the type of a property

If the variable in question is a class property, you can declare a getter method which does the null/undefined check and tightens the type.

class SomeClass {
  private name: string | null;

  public get profileName(): string {
    if (!this.name) {
      throw Error('name should not be null/undefined');
    }
    return this.name;
  }
}

Here the private name field is nullable, but the public profileName property is not.

One downside of this is that you need to come up with two different names for the same field. If you don’t like that, one option is to use the new “private member” syntax:

class SomeClass {
  #name: string | null;

  public get name(): string {
    if (!this.#name) {
      throw Error('name should not be null/undefined');
    }
    return this.#name;
  }
}

Note that the #name syntax is a JavaScript feature, not TypeScript - it works in TypeScript, but the semantics are slightly different than private - check your language reference for details. Also note that you have to use one or the other - you can’t have private on a #private field.

Use a placeholder object

In many cases, if you have a null or undefined value, you can substitute a valid but empty or "blank" object to be used instead. This is especially good for arrays:

(getUserNames() ?? []).find(user => user.name === name);

This is also a great place to use the TypeScript null-coalescing operator (??). Generally you'll only want to use the placeholder if the object is actually null or undefined, not when it's falsey for some other reason.

Push the burden of null/undefined checks onto consuming components

Instead of checking for null right away, modify methods/components to accept null parameters:

const Avatar: FC<{ userId: string | null }> = 
  ({ userId }) => userId 
    ? <div>{getUser(userId)}</div> 
    : null;

const userId: string | null;
return <Avatar userId={userId} /> // Null is OK here since Avatar handles it.

Undefineds and closures

One limitation of the ‘if-block’ technique is that it may not apply if the variable is being used inside a nested function definition:

if (this.userProfile) {
  runInAction(() => {
    // Error! user profile might have changed before this function runs,
    // which means it could be null.
    console.log(this.userProfile.name);
  });
}

You can get around this by saving the value in a local variable:

const lp = this.userProfile;
if (lp) {
  runInAction(() => {
    // OK - lp cannot change because it's a constant.
    console.log(lp.name);
  });
}

Redefine the type containing optional fields

What if you are dealing with a data structure that defined some fields as optional, only you know that those fields are in fact required? This can happen if, for example, the data structure is being generated by some automated process such as a database object mapper. The mapper is being conservative and defining fields as optional, but you know better - and it's a pain to have to check each and every property access for undefined when you know that the data can never be null.

One way to handle this is to redefine the type! There’s a TypeScript function that you can use to make some keys required:

/** TypeScript type modifier thats selectively makes some fields required */
export type RequiredKeys<T, K extends keyof T> = Exclude<T, K> & Required<Pick<T, K>>;

So for example, if we wanted to make UserProfile have required fields:

/** Make optional fields non-optional. */
export type MyUserProfile = RequiredKeys<
  UserProfile,
  'email' | 'persona' | 'valid'
>;

Conclusion

While it may seem like a lot of work to migrate a code base to use strict null checks, it's worth it. The result is higher-quality code that is more robust and resilient against bugs.

Creating Portals in three.js

Talin — Fri, 16 Apr 2021 17:28:55 +0000

In this article, I’ll show you everything you need to know in order to create portals in 3D scenes using the three.js framework.

Portals are probably best known because of the game Portal published by Valve software, but they are useful in all kinds of games and graphics. An example of this is connecting various game levels together, such as a portal that provides a transition between and above-ground and underground environment. Portals can also be used to simulate mirrors and reflective surfaces.

Conceptually, a portal is a kind of in-scene “window” which displays a view into a different part of the virtual world. The portal has two “ends”, one which is the “local” or “source” end that is close to the player’s current location, and a “remote” or “destination” end which represents the location seen through the window. Each end has a separate camera. For purposes of this article, the “primary” camera is the local camera — the one that renders the scene surrounding the portal. The other camera, which we’ll call the “virtual” camera, represents the view within the portal.

Code Organization

Because we may want to have more than one portal in a scene, we’ll need a way to organize the portals which are currently in view. I manage this by defining an ActivePortal class that contains the details for one portal, and then maintain an array of these objects. The ActivePortal class has a number of key methods:

Setter methods to adjust the parameters of the portal, including source position, destination position and aperture size.
An update() method which is called every frame before rendering to compute the on-screen location of the window.
A render() method which is called for each portal after the main scene is rendered.

For a small scene, you can get by with pre-creating ActivePortal instances for every portal in the world. However, for a large game world, you’ll probably want to create them dynamically — adding new ActivePortal instances that are near to the current view position, and removing ones that are too far away.

Synchronized Cameras

One key to making the portal illusion is that the two cameras need to be synchronized in their movements. I like to think of this as working like an old-fashioned pantograph, where the movement of one end causes a corresponding movement in the other end.

We start with the coordinates of each end (note all the code in this article is in TypeScript):

class ActivePortal {
  public readonly sourcePosition = new Vector3();
  public readonly destinationPosition = new Vector3();
}

To calculate the position of the virtual camera, we compute the difference vector between the near and far ends of the portal, and apply that difference to the virtual camera, relative to the primary camera. This calculation is done on every update. Note that both the position and lookAt parameters need to be set.

this.differential
  .copy(this.destinationPosition)
  .sub(this.sourcePosition);

Aperture Size

Another thing we need to know is how big the portal is going to be. Theoretically, a portal doesn’t need to be “flat”, it can be any 3D shape. However, for most purposes you’re going to want a plane surface. For my code, I just use a Box3 to represent the portal aperture, with one of the dimensions set to zero.

Rendering the Aperture

In the primary scene, we’ll render the aperture using a simple box. This box will use a material that renders to the OpenGL stencil buffer. To do this, we’ll need to make a mesh:

this.geometry = new BoxBufferGeometry(
  this.apertureSize.x,
  this.apertureSize.y,
  this.apertureSize.z
);
this.material = new MeshBasicMaterial({
  colorWrite: false,
});
this.mesh = new Mesh(this.geometry, this.material);
this.mesh.geometry.computeBoundingSphere();
this.mesh.frustumCulled = true;
this.mesh.matrixAutoUpdate = false;
this.mesh.renderOrder = 2;
this.mesh.visible = true;
this.mesh.name = 'Portal';

What this mesh is going to do is render a quad shape to the stencil buffer in the primary scene. A later rendering pass will fill in the content of the window.

Note: the reason for setting the renderOrder to 2 is because we want to render the portals after all of the other objects in the primary scene. The reason for this is somewhat non-intuitive: we render the aperture last because we don’t want it to overwrite objects that are in front of the portal. This is because of the stencil buffer: when the aperture is rendered, the depth test is enabled, so the stencil buffer won’t be written in places where there are objects closer to the primary camera than the portal. If we rendered the portals first, then the stencil buffer would be written unconditionally, and any pixels in front of the portal would get overwritten on the subsequent rendering pass.

(Note that if you have translucent objects in the primary scene, you’ll probably want to render them after all of the portals have been rendered, but that is an advanced topic which I won’t cover in this article.)

Because we’re going to be rendering multiple portals, we’ll need a way to distinguish them at the pixel level. This can be done by giving each active portal a unique integer index, and writing that index to the stencil buffer. So the first portal will write the value 1 to the stencil, the second portal uses value 2, and so on. The value of 0 represents the areas of the screen where there is no portal. This theoretically allows up to 255 portals, although it’s unlikely you would want this many because rendering would be very slow.

(There is another way to do portals without using stencils, which is by manipulating the depth buffer — in that technique the portal camera is rendered before the main scene. This technique is simpler in some ways, but it has trouble when you have more than one portal on the scene.)

Although three.js has a way to tell a material to write to the stencil buffer, I needed finer control than what three.js allows. Instead, I use callbacks to set the OpenGL state directly, which also turns out to be simpler and easier to understand than using the three.js API:

this.mesh.onBeforeRender = renderer => {
  if (this.mesh.visible) {
    const gl = renderer.getContext();
    gl.enable(gl.STENCIL_TEST);
    gl.stencilMask(0xff);
    gl.stencilFunc(gl.ALWAYS, this.stencilIndex, 0xff);
    gl.stencilOp(gl.KEEP, gl.KEEP, gl.REPLACE);
  }
};
this.mesh.onAfterRender = renderer => {
  if (this.mesh.visible) {
    // Set everything back to the way it was before
    const gl = renderer.getContext();
    gl.disable(gl.STENCIL_TEST);
    gl.stencilMask(0);
    gl.stencilFunc(gl.ALWAYS, 1, 0xff);
    gl.stencilOp(gl.KEEP, gl.KEEP, gl.KEEP);
  }
};

Computing the screen bounds of the portal

Once we have rendered the primary scene, we need to render each portal as a separate pass. During this pass, we’ll be rendering to a small subset of the screen. This requires computing the 2D bounding box of the aperture. This computation happens in the update() method, before rendering.

const widthHalf = screenSize.width / 2;
const heightHalf = screenSize.height / 2;
this.nearDepth = Infinity;
// Start by making the bounding box empty
this.portalScreenRect.makeEmpty();
const addPortalPoint = (x: number, y: number, z: number) => {
  // Project point to screen space from aperture space
  this.worldPt.set(x, y, z);
  this.worldPt.applyMatrix4(this.mesh.matrixWorld);
  this.worldPt.project(camera);
  // Convert to pixels. Note rounding to prevent jitter.
  this.screenPt.x = Math.round(
    this.worldPt.x * widthHalf + widthHalf);
  this.screenPt.y = Math.round(
    -(this.worldPt.y * heightHalf) + heightHalf);
  // Expand the bounding box to include the point.
  this.portalScreenRect.expandByPoint(this.screenPt);

  // Also track the depth of the nearest aperture corner.  
  // This is used to sort the portals by depth.
  this.worldPt.applyMatrix4(camera.projectionMatrixInverse);
  this.nearDepth = Math.min(this.nearDepth, -this.worldPt.z);
};
// Compute all 8 points of the aperture box and expand the rect.
addPortalPoint(-apertureSize.x, -apertureSize.y, -apertureSize.z);
addPortalPoint(apertureSize.x, -apertureSize.y, -apertureSize.z);
addPortalPoint(-apertureSize.x, apertureSize.y, -apertureSize.z);
addPortalPoint(apertureSize.x, apertureSize.y, -apertureSize.z);
addPortalPoint(-apertureSize.x, -apertureSize.y, apertureSize.z);
addPortalPoint(apertureSize.x, -apertureSize.y, apertureSize.z);
addPortalPoint(-apertureSize.x, apertureSize.y, apertureSize.z);
addPortalPoint(apertureSize.x, apertureSize.y, apertureSize.z);
// Add an extra 2 pixels around the edge of the screen rect to
// account for rounding errors.
this.portalScreenRect.expandByScalar(2);

At this point, we can now determine whether the portal is actually on screen or not:

this.mainScreenRect.min.set(0, 0);
this.mainScreenRect.max.copy(screenSize);
this.isOnscreen = this.mainScreenRect.intersectsBox(
  this.portalScreenRect);
this.mesh.visible = this.isOnscreen;

If the portal is not on screen, we can skip rendering it.

Next, we need to convert the portalScreenRect to a form which can be used to adjust the camera view offset and the scissoring rect. This calculation can also be done in the portal’s update() method:

// Compute viewport coordinates for scissor (Vector4).
this.portalViewport.x = this.portalScreenRect.min.x;
this.portalViewport.y = 
  screenSize.height - this.portalScreenRect.max.y;
this.portalViewport.z = 
  this.portalScreenRect.max.x - this.portalScreenRect.min.x;
this.portalViewport.w = 
  this.portalScreenRect.max.y - this.portalScreenRect.min.y;
this.portalCamera.setViewOffset(
  screenSize.width,
  screenSize.height,
  this.portalScreenRect.min.x,
  this.portalScreenRect.min.y,
  this.portalScreenRect.max.x - this.portalScreenRect.min.x,
  this.portalScreenRect.max.y - this.portalScreenRect.min.y
);

Managing Clip Planes

One common problem with portals is dealing with objects that come between the camera and the portal. For the primary camera, we want this objects to appear on top of the portal. For the virtual camera, however, we don’t. Any objects that are between the virtual camera and the remote end of the portal should be omitted from the scene.

There are a couple of ways to do this — an easy but somewhat inaccurate way to do this is to adjust the near clipping plane of the virtual camera, based on the distance to the portal (which we have already computed).

However, the most robust way to do this is to use three.js’s clipping planes feature.

However, there’s a catch: the number of clipping planes is baked into every three.js shader at compile time. If you change the number of clipping planes, it has to recompile every shader, which is super slow.

The way to get around this is to always have the same number of clipping planes, even in the primary scene. To do this, the primary scene will have a “dummy” clip plane that is far enough away that it doesn’t clip anything.

Rendering the portal content

Because we’re rendering in multiple passes, we don’t want to clear the screen at the beginning of each pass. So we’ll need to clear the color buffer, the depth buffer, and the stencil buffer before rendering the primary scene.

Once the primary scene is rendered (including the apertures, which have now rendered the stencil buffer), we can render the individual portals.

The main rendering loop looks like this:

this.renderer.render(this.scene, this.camera);
this.renderer.getScissor(this.saveScissor);
this.renderer.getViewport(this.saveViewport);
this.renderer.setScissorTest(true);
this.saveClipPlane.copy(this.renderer.clippingPlanes[0]);
portals.forEach(portal => {
  portal.render(this.renderer);
});
this.renderer.setScissorTest(false);
this.renderer.setScissor(this.saveScissor);
this.renderer.setViewport(this.saveViewport);
this.renderer.clippingPlanes[0].copy(this.saveClipPlane);

And the render method of the portal looks like this:

renderer.clippingPlanes[0].copy(this.clippingPlane);
renderer.autoClearStencil = false;
renderer.setScissor(this.portalViewport);
renderer.setViewport(this.portalViewport);
this.mesh.visible = false;
const gl = renderer.getContext();
renderer.autoClearColor = false;
gl.enable(gl.STENCIL_TEST);
gl.stencilFunc(gl.EQUAL, this.stencilIndex, 0xff);
gl.stencilMask(0);
gl.stencilOp(gl.KEEP, gl.KEEP, gl.KEEP);
renderer.render(this.destinationScene, this.portalCamera);
gl.disable(gl.STENCIL_TEST);
this.mesh.visible = this.isOnscreen;

Note once again we’re doing calls directly to GL to set the stencil buffer params. The reason is because three.js doesn’t provide a way to globally set the stencil params for all materials; fortunately three.js doesn’t touch the stencil state unless there is a material that has stencilWrite set to true, which none of my materials do.

Odds and ends

There are a few extra things you can do. The code shown above will render double-sided portals: that is, both the front and the back of the aperture quad will be rendered as a portal. You may not want this, in which case you can test the normal of the quad to see if it’s facing the camera, and skip rendering if it is not.

That is pretty much it. Good luck with your portals!

Note: this article was originally published on medium.com.