Forem: Andrew Morris

Unlocking the True Power of Const with ValueScript

Andrew Morris — Mon, 10 Apr 2023 09:32:20 +0000

On the surface, const allows you to clarify that some variables in your JavaScript program don't change:

const x = 3;
x++; // TypeError 💥

However, this breaks down as soon as you use a non-primitive value:

const x = [1, 2];
x.push(3); // Sure, why not?

There's two ways of thinking about why JavaScript allows this:

const is about which object is bound to x. When you push 3 into the array, x still points to the same object. The array changed, but x did not.
JavaScript has no choice but to allow this, because the current const is already the best consistent behavior it can provide.

To see (2), consider:

const x = getAThing();

const valueBefore = x.value;
someOtherFunction();
const valueAfter = x.value;

valueBefore === valueAfter; // true?

(Complete example with false result.)

It is simply too difficult in JavaScript to determine whether x.value is changed by someOtherFunction(), and therefore it doesn't know whether (deep-style) const has been violated.

(Actually, it's probably fine for simple demonstrations like this one, but this isn't true of the much more complicated examples that quickly emerge in real programs.)

Alternatively, you could also track the const status of the memory itself at runtime, but this would add runtime overhead every time you used const. Large objects could also be composed of many smaller objects with varying const requirements, potentially causing a lot of confusion of its own.

If you want to say const like you mean it, then you might want to avoid writing functions like getAThing() and someOtherFunction() that make changes to shared objects. However, as programs scale, it's very difficult to prevent this. Additionally, there's the mental cost of pursuing this approach.

Enter ValueScript. It's a dialect of TypeScript (and JavaScript) with value semantics. Everything is a value like the 3 from the first example. A 3 is a 3 is a 3. You can increment a variable to make it 4, but that changes the variable. Turning the actual number 3 into 4 would be nonsense.

In ValueScript, the same is true of objects. A [1, 2] is a [1, 2] is a [1, 2]. You can push 3 into it to get [1, 2, 3], but that changes the variable. Turning the actual [1, 2] into [1, 2, 3] would be nonsense.

This is why ValueScript gives you a type error for the second example:

const x = [1, 2];
x.push(3); // TypeError 💥

Knowing whether a method changes the class instance is still a bit tricky, but it's manageable in ValueScript because changes are always local. ValueScript uses a const_subcall instruction for .push, which ensures this cannot be changed inside the call.

ValueScript is able to keep things local by using reference counting and copy-on-write instead of shared references. Otherwise, you could also violate const like this:

export default function main() {
  const leftBowl = ["apple", "mango"];

  let rightBowl = leftBowl;
  rightBowl.push("peach");

  return leftBowl.includes("peach");
  // JavaScript:  true  (even though leftBowl is const)
  // ValueScript: false
}

In JavaScript, leftBowl and rightBowl are the same object, so putting a peach into rightBowl also puts a peach into leftBowl, even though leftBowl is const.

ValueScript doesn't link them together this way. rightBowl can be modified freely, but this has no effect on leftBowl. This is good, because leftBowl is const.

This ability to control variables with const also has benefits beyond programming clarity. It's extremely beneficial that the compiler is able to know when variables change and when they don't. In particular, there's two other places where variables are effectively const even when the const keyword hasn't been used:

Variables that are captured into functions
Imported variables

In these situations, the compiler also uses const_subcall for method calls to ensure they stay constant.

These extensions of const requirements and the enhanced ability to know when variables are constant will also be super beneficial for the future of static analysis in ValueScript. It means the analyzer will know how your program works more often, which will help it optimize, find issues, and reduce binary sizes.

ValueScript is in early development, but it has an extensive playground demonstrating a large subset of JavaScript. It's also open source.

Transactional Try Blocks in ValueScript

Andrew Morris — Thu, 30 Mar 2023 05:14:08 +0000

Exceptions are difficult to get right. A particularly tricky problem that’s easy to overlook is the possibility of partial evaluation inside try blocks.

Consider the following JavaScript program:

let a = new SmallQueue(["item1"]);
let b = new SmallQueue(["item2", "item3", "item4"]);

try {
  const item = a.pop();
  b.push(item);
} catch (e) {
  console.error("Failed to move item from a to b", e);
}

console.log({ a, b });

If 4 items is not too many for SmallQueue, then we can expect an output like this:

{
  a: [],
  b: ["item2", "item3", "item4", "item1"],
}

Otherwise, we’ll log the error from b.push, but unfortunately "item1" will be lost forever:

{
  a: [],
  b: ["item2", "item3", "item4"],
}

This is not good. To fix this, we could check whether b is full first:

let a = new SmallQueue(["item1"]);
let b = new SmallQueue(["item2", "item3", "item4"]);

try {
  if (b.isFull()) {
    throw new Error("b is full");
  }

  const item = a.pop();
  b.push(item);
} catch (e) {
  console.error("Failed to move item from a to b", e);
}

console.log({ a, b });

This will fix the problem, but this type of solution scales poorly. It requires us to understand the exception that might cause partial evaluation and proactively protect against it. The point of exceptions is often to model unexpected things. If you’re not expecting it, then there’s an elevated chance you won’t get this protection code right.

Another option is to save a copy of the variables that might change and restore them on catch:

let a = new SmallQueue(["item1"]);
let b = new SmallQueue(["item2", "item3", "item4"]);

const aCopy = a.copy();
const bCopy = b.copy();

try {
  const item = a.pop();
  b.push(item);
} catch (e) {
  a = aCopy;
  b = bCopy;
  console.error("Failed to move item from a to b", e);
}

console.log({ a, b });

This has potential to scale better, but it has problems of its own:

You might not do it because it’s verbose and feels unnecessarily defensive
It requires a copy operation to be defined
If the copy operation exists, it might not be efficient
If a.pop has side effects, those side effects won’t be reverted

Yet another strategy is to insist that any function call that might throw an exception is individually handled. Sometimes this strategy is enforced at the language level, particularly in languages that don’t have stack-unwinding exceptions, like Go and Rust.

let a = new SmallQueue(["item1"]);
let b = new SmallQueue(["item2", "item3", "item4"]);

let item;

try {
  item = a.pop();
} catch (e) {
  console.error("Failed to move item from a to b", e);
  console.log({ a, b });
  return;
}

try {
  b.push(item);
} catch (e) {
  a.unPop(item);
  console.error("Failed to move item from a to b", e);
}

console.log({ a, b });

Problems with this approach:

Extremely verbose
a.unPop(item) requires us to understand that this operation is needed when b.push(item) fails, which is still easy to miss

Surely there is a better way.

In ValueScript, try blocks are transactional — they either run to completion or they’re reverted.

Our original program does what we wanted without any modifications:

let a = new SmallQueue(["item1"]);
let b = new SmallQueue(["item2", "item3", "item4"]);

try {
  const item = a.pop();
  b.push(item);
} catch (e) {
  console.error("Failed to move item from a to b", e);
}

console.log({ a, b });

{
  a: ["item1"],
  b: ["item2", "item3", "item4"],
}

ValueScript uses the copy-and-restore method, but the problems mentioned earlier do not apply:

1. “You might not do it”
- The ValueScript compiler inserts the instructions automatically
2. “Requires copy operation”
- ValueScript uses value semantics, it just uses aCopy = a, and this is equivalent to copying
3. “copy might not be efficient”
- ValueScript uses copy-on-write all the way down, it’s quite efficient
4. “Side effects of a.pop won’t be reverted”
- ValueScript doesn’t have side effects

I should note that (4) will be weakened in the future, out of necessity. ValueScript doesn’t yet have foreign functions (eg fetch, console.log), but when it does, it won’t be able to revert the side effects of these functions*. You’ll still have to worry about that, but at least ValueScript covers all the internal effects of partial evaluation, so you’ll have more bandwidth to focus on handling the external effects of partial evaluation.

*some awesome foreign functions might include revert operations and these would be automatically called

ValueScript is a dialect of TypeScript with value semantics. It has an online playground including a demonstration of this article’s program, and it’s open source.

How to Speak Binary in TypeScript

Andrew Morris — Thu, 13 May 2021 09:35:23 +0000

If you're sending or receiving data in TypeScript, chances are high that you are using JSON:

const awesomeData = {
  direction: 'up',
  distance: 1000,
};

const json = JSON.stringify(awesomeData);

send(json);

JSON is nice for two reasons:

You can read it ({"direction":"up","distance":1000} is pretty explicit).
JSON doesn't take anything to set up, it's just there.

However, this simple utility doesn't play very nicely with TypeScript:

const json = receive();

// { direction:'up', distance: 1000 }
const awesomeData = JSON.parse(json);

// TypeScript compiler: "Sure, why not?"
awesomeData.direction = 1;

JSON.parse is one of those APIs that opts out of type checking - it returns any, which means TypeScript knows nothing about it, but will let us do anything with it.

That's not cool. We're using TypeScript to avoid exactly this problem.

There is a better way:

import * as tb from 'typed-bytes';

const Awesome = tb.object({
  direction: tb.enum_('up', 'down', 'left', 'right'),
  distance: tb.size,
});

type Awesome = tb.TypeOf<typeof Awesome>;

function sendData() {
  const awesomeData: Awesome = {
    direction: 'up',
    distance: 1000,
  };

  // Here encodeBuffer knows that awesomeData needs to match type Awesome
  const buffer = tb.encodeBuffer(Awesome, awesomeData);

  send(buffer);
}

function receiveData() {
  const buffer = receive();

  const awesomeData = tb.decodeBuffer(Awesome, buffer);

  // Error: Type '1' is not assignable to type '"up" | "down" | "left" | "right"'
  awesomeData.direction += 1;
}

Hooray, now awesomeData is correctly type checked 🎉.

It's not just that though. Take a closer look at buffer:

ArrayBuffer { [Uint8Contents]: <00 e8 07>, byteLength: 3 }

Three bytes. That's all we needed. In JSON we used 34 bytes.

Why not just use Protocol Buffers?

I'm glad you asked. typed-bytes is a public domain library that lives at typedbytes.org. There's a detailed comparison over there 😉.