When machine learning runs on your server, debugging is familiar territory: you have logs, profilers, stack traces, and a machine you control. When the same model runs inside a visitor's browser, every assumption changes. The runtime is a different GPU, CPU, or mobile browser each time. Memory is constrained. WebGL, WebGPU, and WASM backends behave differently. And once the page is closed, all evidence disappears with it.

Debugging in-browser ML therefore needs three layers of defense. First, profile the inference path so you know where time and memory actually go. Second, log structured events so failures are reconstructable after the fact. Third, build graceful fallbacks so the worst outcome is a slower answer, never a blank page.

Profile the Inference Path First

Do not guess where the latency is. Measure each stage separately: model download, weight deserialization, session creation, input preprocessing, and session.run itself. A single performance.now() around the whole pipeline will hide the stage that regressed.

const t0 = performance.now();
const output = await session.run(inputs);
const elapsed = performance.now() - t0;
logEvent("inference", {
  elapsed,
  backend: session.handler,        // e.g. "webgpu", "webgl", "wasm"
  inputTokens: inputs.shape[1]
});

A few findings this pattern typically surfaces:

  • The first inference on a backend is always slower than the rest. Warmup costs (shader compilation, WebGPU pipeline creation) can dwarf steady-state inference by an order of magnitude.
  • Backend selection matters more than model size. A 22 MB quantized ONNX model on a slow WASM fallback can take longer per query than the same model on WebGL, even though the bytes are identical.
  • Feature extraction and post-processing are frequently slower than the model itself. Tokenization, padding, and top-k selection all show up here.

The performance techniques ReLU.chat documents — query embedding memoization, bounded top-k selection instead of full sorts, LRU caches, and pre-built BM25 IDF values — are exactly the output of this profiling exercise. Every one of them is a profiler finding turned into a code change.

Memory is the other half of profiling. performance.memory is only available in Chromium, so track what you can: the byteLength of your typed arrays, the sizes of tensors returned by each session.run, and the growth of caches across turns. A common failure is unbounded caching — embeddings and retrieval results stored per turn with no eviction policy. ReLU.chat bounds this with LRU caches, query embedding memoization, and a 30-turn session window. Without the same discipline, memory grows linearly with conversation length until the tab dies.

Log Structured Events, Not Console Log Soup

console.log strings are fine for interactive debugging and useless for everything else. What you want is a small ring buffer of structured events, each with a timestamp, a stage name, and a few fields. Keep the buffer on-device; a privacy-first product with no tracking should never ship telemetry that leaves the browser.

const eventBuffer = [];
function logEvent(stage, fields) {
  eventBuffer.push({ stage, ts: performance.now(), ...fields });
  if (eventBuffer.length > 500) eventBuffer.shift();
}

Correlate events across stages with a single turn ID so a user's message produces one trace: intent classification, retrieval, policy inference, generation, render. When a user reports "it answered instantly but the answer was wrong," the trace tells you which stage produced the output. When they report "it hung for ten seconds," the trace tells you where the gap is.

The browser DevTools Performance panel is the second half of this story. Because everything runs locally, every model load and inference appears as a flame chart entry you can inspect without any server access.

The Three-Stage Pipeline Is a Built-In Debugger

Progressive loading — ReLU.chat loads in three stages: a heuristic/BOW fallback for instant first turns, then the MLP policy, then a hot-swap to the full dense MiniLM model — is a feature, but it is also a diagnostic instrument. Every response can be tagged with the stage that produced it:

  • Stage 1 answer when stage 3 was expected: the ~22 MB model failed to load or the service worker cache missed.
  • Stage 2 answer when stage 1 was expected: the policy weights loaded, so the fallback was skipped as designed.
  • Full model answer: the normal path, and the one to benchmark against.

Tagging responses with their producing stage turns user bug reports into precise signals. "It started answering weird after a few turns" becomes "stage 1 was answering for the whole session," which points straight at a weight-load failure rather than a ranking bug.

Build Graceful Fallbacks for Every Failure Mode

Assume every load can fail and every numeric result can be nonsense. Design the degradation chain in advance:

  • Weight load failure falls back to the 15 parameterized heuristic decision thresholds used during cold start, so the chat still functions with rule-based behavior.
  • Missing or corrupt cached weights trigger a fresh download instead of a crash.
  • NaN checks on embeddings catch silent numeric corruption before it poisons cosine similarity. A single NaN in a 384-dimensional vector makes every downstream comparison meaningless.
  • Feature normalization should clip and scale defensively, because a NaN or Infinity in one input feature can propagate through an MLP to every action head.
// Cheap finite check before any similarity computation
if (!Number.isFinite(embedding.reduce((s, v) => s + v, 0))) {
  return fallbackTo("stage-1", "non-finite embedding");
}

A useful mental model: the fallback chain should be a ladder, not a cliff. Full model, then quantized policy, then heuristics, then a polite plain-text answer. Each rung trades quality for reliability, and no rung should ever render a blank screen.

Key Takeaway

In-browser ML fails differently from server ML, so it must be debugged differently: profile each inference stage separately, keep structured on-device event logs correlated by turn, use the progressive-loading stage tags as a diagnostic signal, and always have a graceful fallback ladder so a model failure degrades to a slower answer instead of a broken page. Measure first, log second, and assume the model will sometimes not load at all.