Forem: JairusSW

Quickly detecting JSON Escapes with SWAR

JairusSW — Wed, 01 Apr 2026 05:27:14 +0000

Most JSON string data is boring.

It is just ASCII text that does not contain " or \, does not dip below 0x20, and does not force any special-case escaping at all.

The annoying part is that a serializer still has to prove that.

The naive loop is obvious:

for (let i = 0; i < src.length; i++) {
  const code = src.charCodeAt(i);
  if (code == 34 || code == 92 || code < 32) {
    // escape
  }
}

That is correct, but it is also one branch per code unit in the hottest part of the serializer.

For json-as, I wanted the fast path to ask a cheaper question:

Does this whole chunk contain anything interesting at all?

That is where SWAR fits nicely.

For this post, the companion code is here:

What needs detecting?

When serializing a JSON string, these are the lanes that matter:

" because it must become \"
\ because it must become \\
control characters < 0x20
non-ASCII UTF-16 code units, because they cannot stay on the pure ASCII copy path

If I can classify four UTF-16 lanes at once inside one u64, then the common case becomes:

load 8 bytes
compute a mask
copy directly if the mask is zero

That is the whole job of this helper written in the AssemblyScript Language

@inline function detect_escapable_u64_swar_safe(block: u64): u64 {
  const lo = block & 0x00ff_00ff_00ff_00ff;
  const ascii_mask =
    ((lo - 0x0020_0020_0020_0020)
      | ((lo ^ 0x0022_0022_0022_0022) - 0x0001_0001_0001_0001)
      | ((lo ^ 0x005c_005c_005c_005c) - 0x0001_0001_0001_0001))
    & (0x0080_0080_0080_0080 & ~lo);

  const hi_mask =
    ((block - 0x0100_0100_0100_0100) & ~block & 0x8000_8000_8000_8000)
    ^ 0x8000_8000_8000_8000;

  return (ascii_mask & (~hi_mask >> 8)) | hi_mask;
}

It looks cryptic, but it is just two independent predicates packed into one result.

Why UTF-16 makes this convenient

AssemblyScript strings are UTF-16, and that matters a lot here.

For plain ASCII text:

the low byte of each 16-bit lane contains the character
the high byte is zero

That gives a very convenient layout. I can inspect the low bytes for JSON escape cases, and separately inspect the high bytes to notice when the block is no longer plain ASCII.

So this is not a universal string trick. It is a particularly good fit for UTF-16 data.

The low-byte test

First, isolate the low byte of each UTF-16 lane:

const lo = block & 0x00ff_00ff_00ff_00ff;

Now I only care about three low-byte predicates:

< 0x20
== 0x22
== 0x5c

The detector builds those without branches:

const ascii_mask =
  ((lo - 0x0020_0020_0020_0020)
    | ((lo ^ 0x0022_0022_0022_0022) - 0x0001_0001_0001_0001)
    | ((lo ^ 0x005c_005c_005c_005c) - 0x0001_0001_0001_0001))
  & (0x0080_0080_0080_0080 & ~lo);

This is standard SWAR arithmetic:

subtraction to manufacture per-lane underflow or equality signals
masking to collapse those signals into lane-local high bits
no per-character branch in the hot path

If one of the four low bytes looks special, its lane gets marked in ascii_mask.

The high-byte test

That still leaves non-ASCII code units.

The ASCII trick only works when every high byte is zero, so the detector also asks:

Is any UTF-16 high byte non-zero?

That is what this part does:

const hi_mask =
  ((block - 0x0100_0100_0100_0100) & ~block & 0x8000_8000_8000_8000)
  ^ 0x8000_8000_8000_8000;

The result is a mask over the high bytes of the four lanes. If a lane is not plain ASCII, that lane is marked.

Then the final merge is:

return (ascii_mask & (~hi_mask >> 8)) | hi_mask;

That means:

keep the ASCII escape hits for truly ASCII lanes
also mark any lane whose high byte is non-zero

The serializer now has exactly the answer it needs:

Which lanes force me off the pure ASCII copy path?

The unsafe variant

There is also a cheaper detector:

@inline export function detect_escapable_u64_swar_unsafe(block: u64): u64 {
  const lo = block & 0x00ff_00ff_00ff_00ff;
  const ascii_mask =
    ((lo - 0x0020_0020_0020_0020)
      | ((lo ^ 0x0022_0022_0022_0022) - 0x0001_0001_0001_0001)
      | ((lo ^ 0x005c_005c_005c_005c) - 0x0001_0001_0001_0001))
    & (0x0080_0080_0080_0080 & ~lo);

  return ascii_mask | (block & 0xff00_ff00_ff00_ff00);
}

This one does not prove that a non-ASCII lane is really dangerous. It just marks any lane with a non-zero high byte and lets the later slow path sort it out.

That means more false positives, but a smaller and faster detector.

So the tradeoff is simple:

safe: do more proof up front
unsafe: bail out faster and prove less on the hot path

If your strings are mostly ASCII, that is often a good deal.

What the serializer gets from this

Once you have the mask, the fast path becomes tiny:

while (true) {
  const block = load<u64>(srcStart);

  let mask = detect_escapable_u64_swar_unsafe(block);

  while (mask != 0) {
    const laneIdx = ctz(mask) >> 3;
    // clear low and high bytes
    mask &= ~(0xffff << (laneIdx << 3));
    // even (0 2 4 6) -> confirmed ascii escape
    // odd (1 3 5 7) -> possibly a unicode code unit or surrogate

    if ((laneIdx & 1) === 0) {
      // handle non-surrogates (fast path)
      continue;
    }

    // and handle surrogates here (unlikely to be hit)
  }

  offset += 8;
}

That is the payoff.

For ordinary ASCII blocks with no escapes, the serializer just copies 8 bytes and moves on.

Only when the mask is non-zero does it need to identify the exact lane and expand the escape sequence.

That shifts the cost model in the right direction. Instead of repeatedly asking “does this character need escaping?”, the hot path asks “is there anything interesting in this whole block?”

Benchmarking safe vs unsafe

I reran the detector-only benchmark with matching harness shapes in both implementations:

AssemblyScript compiled to Wasm and run with wasmer --llvm --enable-pass-params-opt
native C compiled with aggressive host-tuned flags and LTO

Both versions use the same payload size, warmup policy, operation count, and logging structure. The benchmark code is here:

The two payload shapes are:

plain ASCII text with no escapes
escape-heavy text with quotes, backslashes, and control characters

Here are the current numbers from that harness:

Case	Wasm / `wasmer --llvm --enable-pass-params-opt`	Native C
safe / plain	5030 MB/s	6824 MB/s
unsafe / plain	6355 MB/s	8562 MB/s
safe / escaped	4908 MB/s	6295 MB/s
unsafe / escaped	7093 MB/s	8331 MB/s

That is exactly what I wanted to know. The unsafe detector is not just theoretically smaller. It buys measurable throughput in the actual runtime I care about. When integrating it into an actual JSON serializer, it's marginally faster too.

If you want to rerun it, the example folder is here:

01-c-and-wasm-benchmark

And the commands are just:

make install
make build
make run

Looking at the generated code

The throughput numbers tell you the tradeoff is real, but the code shape explains why.

The unsafe detector compiles to a smaller body because it skips the extra high-byte proof/merge logic from the safe version. That is easiest to see in the inspection example:

02-assemblyscript-inspection

There is nothing magical there. The unsafe version is just doing less work.

If you want to see the real implementation in context, these are the relevant json-as sources:

Quickly parsing Unicode Escapes with SWAR

JairusSW — Mon, 05 Jan 2026 22:16:14 +0000

Last night I went down a rabbit hole optimizing Unicode escape decoding in a JSON parser using SWAR (SIMD Within A Register) techniques.

Even though unicode escapes don't occur often on average, making it a cold path, I still wanted to see how far a purely arithmetic approach could be pushed. I considered using a LUT (LookUp Table), but it took more cpu cycles than ideal. Eventually though I stumbled upon this stunningly beautiful little function:

(c & 0xF) + 9 * (c >> 6)

It may seem insignificant, but this function maps ASCII hex characters (0-9, A-F, a-f) to their numeric values without branches or tables.

My use case though, was to parse unicode escape sequences in JSON quickly and efficiently, meaning that I'd have to run this function for each nibble in \uXXXX, which makes it a computational drain.

Instead of converting each character one-by-one with the previous function, I can utilize SIMD Within A Register. SWAR lets you run the same logic on all four nibbles in parallel inside a single 64-bit register, meaning that I can easily parallelize this to:

// Convert a single \uXXXX -> u16. Assume UTF/WTF-16
@inline function hex4_to_u16_swar(block: u64): u16 {
  // (c & 0xF) + 9 * (c >> 6)
  // Perform the above function for all nibbles in parallel
  block = (block & 0x0F000F000F000F)
    + ((block >> 6) & 0x03000300030003) * 9;

  // Pack each nibble into a u16
  return <u16>(
    ((block >> 0)) << 12 |
    ((block >> 16)) << 8 |
    ((block >> 32)) << 4 |
    ((block >> 48))
  );
}

Now, I can parse JSON strings at over 7GB/s. Hooray! 🎉

Low-level tricks aren't always necessary, given the excess of computing power we live in today, but when they do apply, they sure are hard to beat.

You can view the source here: https://github.com/JairusSW/json-as

So, you can benchmark in AssemblyScript...

JairusSW — Sat, 14 May 2022 22:18:36 +0000

Introducing... Astral--A minimalistic benchmarking library for AssemblyScript. Based off of criterion.rs.
With this new benchmarking tool, we are able to easily visualize and optimize performance in our code precisely. Usually, we would have to use a while loop and Date.now or performance.now bindings to measure performance. Astral adds warmup periods, logging, and a few other tools. Lets get started!
In an existing AssemblyScript project, install as-tral.

npm i -D @as-tral/cli

Next, create a benches folder in your assembly directory. Add a file named as-tral.d.ts in there.

assembly/
└── __benches__/
    └── as-tral.d.ts

In as-tral.d.ts, copy and paste

/// <reference path="../../node_modules/@as-tral/cli/as-tral.d.ts" />

Stick any file with a .ts extension in benches. You can even have multiple.

assembly/
└── __benches__/
    ├── as-tral.d.ts
    └── my-benchmark.ts

Your benchmark will live in this file. Let's create an example benchmark.

// to ensure accurate benchmarks, we must make sure that binaryen doesn't do any sneaky
// optimizations on our input without us knowing. Thus, we must use `blackbox`.
const input = blackbox("The quick brown fox jumped over the lazy dog.".repeat(10));

// our string here must be a compile time constant.
// open an issue if you'd like to see this constraint lifted.
bench("string split", () => {
    // this function body will be run many times.
    // we must make sure our compiler won't throw away the computation,
    // so we use `blackbox` here again.
    blackbox(input.split(" "));
});

Now, let's run as-tral.

rom@i9-cabin:~/demo$ npx astral
Compiling assembly/__benches__/hello.ts

Benchmarking string split: Warming up for 3000ms
Benchmarking string split: Collecting 100 samples in estimated 5018.6ms (1.2M iterations)
Benchmarking string split: Analyzing
string split            time: [3770.9ns 3775ns 3778.9ns]
Found 4 outliers among 100 measurements (4%)
  3 (3%) low mild
  1 (1%) high mild

Pretty great!

JSON in AssemblyScript

JairusSW — Tue, 20 Jul 2021 14:44:53 +0000

Finally, AssemblyScript has a fully-functional JSON implementation. as-json implements full JSON compatibility that enables AssemblyScript to use JSON to communicate with APIs, store data, and more. In this article, I will introduce you to as-json and help you include it into your project.

Getting Started

as-json uses the same general API as the JavaScript JSON object. We can then use it just like JS. However, we must install and prep it first.

npm install json-as

--transform json-as/transform

Installation complete. Lets use some JSON! 🔥

json-test.ts

import { JSON } from 'json-as'

// Create the Schemas
// @ts-ignore
@json
class JSONSchema {
  firstName: string;
  lastName: string;
  human: boolean;
  age: i32;
  meta: Meta;
  language: string;
  location: f64[];
}

// @ts-ignore
@json
class Meta {
  country: string;
  awesome: boolean;
}

// Create the JSON object
const data: JSONSchema = {
  firstName: 'Jairus',
  lastName: 'Tanaka',
  age: 14,
  human: true,
  meta: {
    country: 'US',
    awesome: true
  },
  language: 'english',
  location: [-43.130850291, 32.926401705]
};

// Now, encode and decode
const encoded: string = JSON.stringify(data)
console.log(`Encoded: ${encoded}`)
const decoded = JSON.parse<JSONSchema>(encoded)
console.log(`Decoded:`)
console.log(`{`);
console.log(` firstName: ${decoded.firstName},`);
console.log(` lastName: ${decoded.lastName},`);
console.log(` age: ${decoded.age},`);
console.log(` human: ${decoded.human},`);
console.log(` meta: {`);
console.log(`   country: ${decoded.meta.country},`);
console.log(`   awesome: ${decoded.meta.awesome}`);
console.log(` },`);
console.log(` language: ${decoded.language}`);
console.log(` location: [${decoded.location[0]}, ${decoded.location[1]}]`);
console.log(`}`);

Note: If you are not running in WASI, use as-console instead.

So, json-as serialized and deserialized JSON. Lets see if it was correct.

json-test.ts

import { JSON } from 'json-as'

// Create the Schemas
// @ts-ignore
@json
class JSONSchema {
  firstName: string;
  lastName: string;
  human: boolean;
  age: i32;
  meta: Meta;
  language: string;
  location: f64[];
}

// @ts-ignore
@json
class Meta {
  country: string;
  awesome: boolean;
}

// Create the JSON object
const data: JSONSchema = {
  firstName: 'Jairus',
  lastName: 'Tanaka',
  age: 14,
  human: true,
  meta: {
    country: 'US',
    awesome: true
  },
  language: 'english',
  location: [-43.130850291, 32.926401705]
};

// Now, encode and decode
const encoded: string = JSON.stringify(data)
const decoded = JSON.parse<JSONSchema>(encoded)
// We perform an equality check
if (encoded == JSON.stringify(decoded)) {
  console.log('Yay! JSON-AS works! 😄')
} else {
  console.log('Oof. JSON-AS died.😖')
}

Yay! JSON is now working for AssemblyScript. Go ahead and mess around with it and let me know what you make.
With the up-and-coming release of JSON-AS v0.2.0, full dynamic objects and arrays will be supported.
NPM: https://www.npmjs.com/package/json-as
GitHub: https://github.com/aspkg/as-json
Thanks for reading 😁

What is AssemblyScript?

JairusSW — Mon, 12 Jul 2021 18:34:40 +0000

AssemblyScript — a variant of TypeScript that compiles to WebAssembly. Since it is compiled, it outperforms JavaScript in some cases. AS follows the TypeScript syntax as closely as possible and even adds more features to it. AssemblyScript allows us to write fast WebAssembly for the Web and Server without learning another language.

AssemblyScript is very easy to use. All you need to do is install it via NPM.

Try it out online: Online editor

~ npm i assemblyscript --save-dev
~ npx asinit .
~ npm i

What did that do? First of all, it installed both the loader and the compiler. Secondly, it made a template project with an add function. Now, we just need to compile it to WebAssembly.

~ npm run asbuild

So, if you check out the /build folder, there are the .wasm files that were built. NodeJS and JavaScript both provide a way to run WebAssembly files and AssemblyScript provides its own loader to work with the code. To start our code, we need another file

test.js

const wasmModule = require('./index')
// This works just like a normal module
console.log(wasmModule.add(2,9))
// -- 11

Now, run it!

~ node test.js

It should have outputted the number 11.

JavaScript code for the add function would look like this:

function add(a, b) {
    return a + b
}
module.exports = {
    add: add
}

The AssemblyScript code looks like this:

export function add(a: i32, b: i32): i32 {
    return a + b;
}

Pretty similar, right? When we compile it, we can require it just like a normal JavaScript file. Keep in mind that WebAssembly is sandboxed which means it can’t access the system, make HTTP requests, or log to the console. However, AssemblyScript supports both WASI and JS bindings (calling JS from AS).

If you have any questions or comments, feel free to comment or join the AssemblyScript Discord. Or, check out the website.😉

P.S: There is a tutorial at https://jtanaka.gitbook.io/guide/