JavaScript has one number type: IEEE 754 double-precision, float64. Machine learning libraries do not share that preference. ONNX Runtime tensors default to float32, transformers.js embeds arrive as float32, WebGPU shaders compute in float32, and quantized models go far lower. Every time a float64 JavaScript number meets a float32 tensor, a conversion happens, and with it a small, silent loss of precision.
The engineering question is not "which is more precise" — float64 always is. The question is where the extra precision is worth its cost, and where float32 (or int8) is not just acceptable but preferable. This post walks through the trade-off in the context of in-browser NLP systems like ReLU.chat.
JavaScript Numbers Are Float64, ML Frameworks Are Not
Number in JavaScript is a double. But the moment you work with model tensors you leave Number behind and enter typed-array territory, where the element size is explicit:
const f32 = new Float32Array(384); // 4 bytes per element
const f64 = new Float64Array(384); // 8 bytes per element
Indexing into a typed array converts values to that array's precision. Float32Array reads and writes round to float32, so a cosine similarity computed over float32 embeddings is a float32 computation no matter how precise the surrounding JavaScript is. This is usually what you want: it matches what the model produced and what the inference runtime computes.
The float64-vs-float32 distinction also shows up in API design. performance.now() returns float64. IndexedDB stores doubles. The practical rule: keep computations in the precision of the data they consume, and reserve float64 for accumulators and derived scores where error compounds.
Where Float32 Is Plenty
For most in-browser ML work, float32 is not a compromise; it is the right precision. Consider cosine similarity over embeddings. all-MiniLM-L6-v2 produces 384-dimensional embeddings, and ReLU.chat computes dense cosine similarity against them. The inputs are normalized vectors whose components sit in a modest range. A float32 dot product of 384 terms accumulates relative error on the order of 1e-7 — far below the noise floor of the model itself, where two paraphrases of the same sentence routinely differ by more than 0.01 in cosine distance.
The same holds for softmax-style scoring. Intent classification here compares cosine similarity against 19 prototypes per category, blends best-vs-average 70/30, and calibrates with a temperature of 1.5 softmax. The temperature exists to shape the distribution, and it is far more tolerant than the 1e-7 error float32 introduces.
Rule of thumb: if the downstream decision is a ranking or an argmax over a small set of candidates, float32 error is irrelevant. If the downstream computation is an iterative accumulator, read on.
Where Float64 Earns Its Keep
Iterative updates are where float32 silently drifts. ReLU.chat maintains an EMA summary vector of the conversation with alpha = 0.75, and entity scores that decay with a half-life of 5 turns. Both are repeated applications of the same update:
summary = alpha * summary + (1 - alpha) * newEmbedding;
Each application rounds to the accumulator's precision. In float32, a value near zero loses relative precision quickly — the infamous "small value added to a large value" problem — so over 30 turns of updates, the summary vector's tail components drift more than the model's own error budget. The fix is trivial: keep the accumulator in float64 and convert once at the end.
let acc = 0.0; // float64 accumulator
for (const x of scores) acc += x;
The same reasoning applies to any sum over many terms: importance scores accumulated across a 30-turn window, BM25 term scores summed over query terms, and relevance score totals. If you sum a hundred float32 values, the error can exceed one float32 ulp. Summing in float64 and rounding once costs nothing measurable and removes an entire class of "the ranking is unstable" bugs.
Memory and Speed Budgets
The cost of float64 is concrete. A float32 tensor of 1 million elements takes 4 MB; the same tensor in float64 takes 8 MB. In a browser tab sharing memory with the page, the DOM, and the service worker, that difference is real.
Quantization makes the point sharper. ReLU.chat's policy MLP — 25 inputs to 128 to 64 to 6 action heads, about 13,079 parameters — is auto-quantized to symmetric int8 at load time, a roughly 4x memory reduction versus float32. The 22 MB MiniLM ONNX model is already quantized to ship that small. Neither would fit the same budget at float64.
Speed is more nuanced. Native WASM and WebGPU backends are optimized for float32; float64 compute is often emulated and slower. But in JavaScript, typed-array float64 arithmetic is fast, and the conversions at tensor boundaries are the real cost. The efficient pattern is: float32 tensors for model I/O, float64 accumulators for derived statistics, int8 for storage of weights, and never mix precisions inside a hot loop without measuring.
Measure rather than assume. The conversion between a float64 JavaScript number and a float32 tensor element is a single typed-array write, but copying a whole tensor between precisions costs memory bandwidth proportional to its size. A 384-dimensional embedding copy is about 1.5 KB — noise. The same copy for a 22 MB model would be a visible pause on a mid-range phone. Keep conversions at the boundaries of small tensors, and never round-trip a large tensor through a different precision out of caution:
// Quantize weights to int8 at load; keep activations in float32
const scale = 127 / Math.max(...weights.map(Math.abs));
const int8 = Int8Array.from(weights, (w) => Math.round(w * scale));
Key Takeaway
Use float32 for anything that feeds a model or comes out of one — embeddings, activations, similarity scores — because it matches the runtime and its error is below the model's noise floor. Reserve float64 for accumulators and iterative updates like EMA summaries and decayed entity scores, where rounding error compounds across turns. Let int8 quantization shrink weights in memory, and remember that every precision decision is a memory and speed decision too.