JavaScript is excellent at UI and orchestration, but the inner loops of machine learning — matrix multiplication, attention, tokenization — are numeric kernels that want a compiled language. WebAssembly (WASM) is the browser's answer: a portable binary format that runs C, C++, and Rust code at near-native speed, sandboxed, in every major browser. It is the reason transformer models can execute entirely in a tab instead of on a server.

What WebAssembly Is

WebAssembly is a binary instruction format for a stack-based virtual machine with linear memory. Modules are loaded and instantiated from JavaScript, and their exported functions are callable directly:

const result = await WebAssembly.instantiateStreaming(
  fetch("kernels.wasm"),
  importObject
);
const add = result.instance.exports.add;
console.log(add(2, 3)); // 5

The design matters for ML in three ways. There is no garbage collector, so memory behavior is deterministic and predictable. Execution is sandboxed, with no direct access to the host beyond explicit imports. And the semantics are specified precisely, which is what allows engines to compile WASM to native code ahead of time. You write kernels in C or Rust, compile with clang/emscripten or a Rust WASM target, and ship one binary that runs identically on Windows, macOS, Android, and iOS browsers.

From asm.js to WASM

WASM was not the first attempt to run compiled code in the browser. Emscripten pioneered asm.js, a restricted subset of JavaScript that engines could recognize and compile aggressively. asm.js worked, but it was still JavaScript: it had to survive parsing as JS syntax, which imposed overhead and awkwardness. WebAssembly replaced it with a real binary format and a dedicated compilation pipeline, giving engines a faster-to-decode input and better optimization targets. The lineage matters because it explains the ecosystem: the toolchains (Emscripten, LLVM wasm targets) and the design philosophy (compiled kernels, thin JS glue) carried straight over.

Why It Is Faster Than Plain JavaScript for Numeric Loops

Modern JavaScript engines JIT-compile typed-array code surprisingly well, but numeric kernels still pay costs that WASM avoids: bounds checks, boxing of intermediate values, and garbage collection pressure from allocation-heavy loops. WASM operates directly on raw linear memory with explicit loads and stores, so the optimizer can keep values in registers and unroll loops aggressively.

The bigger win is SIMD. WASM SIMD (the v128 instruction set) processes four 32-bit floats per instruction. A 384-dimensional dot product — the core operation in embedding-based retrieval — becomes roughly 96 vector instructions plus a reduction, instead of 384 scalar operations. Combined with WASM threads over SharedArrayBuffer, CPU-heavy inference kernels can also use multiple cores when the device has them:

// pseudo-code: SIMD dot product over Float32Array a and b
let sum = 0;
for (let i = 0; i < a.length; i += 4) {
  // v128 loads, f32x4 multiply, f32x4 add
  sum += dot4(a.subarray(i, i + 4), b.subarray(i, i + 4));
}

The effect is that an optimized WASM matrix multiply approaches the performance of native code on the same CPU — something JavaScript rarely achieves. A tiny kernel in C makes the point:

float dot4(const float* a, const float* b) {
  float sum = 0.0f;
  for (int i = 0; i < 4; i++) sum += a[i] * b[i];
  return sum;
}

Compiled with clang --target=wasm32 -msimd128, that loop becomes a handful of v128 instructions, and a JS wrapper can call it thousands of times per query.

ML Runtimes Are Compiled to WASM

The entire on-device ML ecosystem is built on this. ONNX Runtime ships a WASM backend so ONNX models run in the browser unchanged. transformers.js wraps tokenizers and models compiled to WASM so the Hugging Face pipeline API works client-side. Tokenizer libraries like SentencePiece compile from C++ to WASM. In each case the pattern is the same: hard numeric work in compiled code, thin JavaScript glue on top.

ReLU.chat follows exactly this pattern. Its retrieval model — all-MiniLM-L6-v2, quantized to ONNX at about 22 MB — runs in the browser through ONNX Runtime and transformers.js, producing 384-dimensional embeddings without any server. The tokenizer runs as compiled code too, which is why tokenizing a query adds negligible latency. A service worker pre-caches the model and policy weights so the WASM modules are already on disk for subsequent visits.

Where WASM Sits in the On-Device Stack

Think of browser inference as three tiers:

  • JavaScript: glue, UI, session logic, and small math.
  • WASM: portable CPU kernels — inference, quantization, tokenization. Works everywhere, no GPU required.
  • WebGPU: GPU kernels when available, with much higher throughput for large models.

WASM is the reliable baseline: it runs on any device with a browser, including machines with no GPU and locked-down enterprise hardware. The costs are real but manageable: the runtime adds a few megabytes of download, modules need compile/instantiation time on first load, wasm32 memory is capped at 4 GB, and execution is single-threaded unless you explicitly add threads. When a GPU is available and the model is large, WebGPU takes over — but WASM remains the portability floor.

Progressive loading is how you hide the costs. ReLU.chat loads in three stages: a heuristic fallback that answers the first turns instantly, then the MLP policy network, and finally the full dense MiniLM model hot-swapped in — while the service worker pre-caches the model and policy weights. Users get immediate responses; the heavy WASM modules load in the background and take over when ready.

Key Takeaway

WebAssembly gives browser AI compiled, near-native CPU performance with a portable and sandboxed runtime, and it is the substrate under every serious on-device inference stack — from ONNX Runtime to transformers.js. Paired with progressive loading and service-worker caching, it is what lets a transformer model run fully inside a browser tab.