Most chat products are built online-first: the client sends a message, a server responds, and if the network drops the client shows an error. Offline-first inverts this. The application is designed so that the network is never a hard dependency, and the primary experience works with zero connectivity.

For browser-based, on-device chatbots this inversion is not a compromise — it is the natural architecture. ReLU.chat runs no servers, uses no LLMs, and tracks nothing. All state lives in the browser. The offline path is not a degraded mode; it is the only mode, and it forces a design that is simpler and more robust than a networked one.

Offline-First Is the Default for Browser Chat

An on-device chatbot has no server to reach, so "offline" simply describes the steady state. The practical consequences:

  • Model weights must be available locally. A service worker pre-caches the model and policy weights so the first visit after install does not depend on a live fetch.
  • Session state must survive without a backend. Conversations, entity mentions, and summary vectors live in IndexedDB, not in a cloud database.
  • There is no retry-until-server-responds logic, because there is no server. Failures come from the device itself: storage full, model not loaded, worker terminated by the browser.

Once you internalize that the device is the only infrastructure, every design decision follows. Memory is your server budget, IndexedDB is your database, and the main thread is your scarce CPU.

A useful exercise is to simulate the failures before they happen. DevTools network throttling does little for a fully offline app, but the offline checkbox, storage-pressure reloads, and slow-CPU emulation reveal real failure modes: a service worker serving stale weights, a queue that deadlocks while the worker thread is busy, an IndexedDB write that fails when storage is full. Each is a distinct bug with a distinct fix, and each is cheaper to find in development than in a user's browser.

Model Every Turn as a Queue of Work Items

A chat turn is a pipeline: tokenize and embed the query, run intent classification, retrieve candidates, score and rank, run the policy network, generate the response, render it. If you implement this as one synchronous function, you cannot recover from a mid-pipeline failure, and you cannot observe where time went.

Model the pipeline as a queue of discrete work items. Each item is small, resumable, and persisted.

const turnQueue = new WorkQueue("turn-pipeline", {
  concurrency: 1,          // turns are serial: responses must not interleave
  persist: true            // survive page reloads via IndexedDB
});

await turnQueue.enqueue({ type: "embed", text: userMessage });
await turnQueue.enqueue({ type: "retrieve", queryId });
await turnQueue.enqueue({ type: "generate", context: ctx });

A persisted, serial queue buys you three things. First, crash recovery: if the page reloads mid-turn, the queue is re-read on boot and the remaining items finish. Second, observability: each item can log its duration and result. Third, backpressure: if the embed model is still downloading, the queue simply waits instead of firing a failed request.

The queue also naturalizes the progressive-loading design: heuristic/BOW fallback answers the first turns instantly, the MLP policy takes over once its weights are in memory, and the full MiniLM model hot-swaps in later. Each stage is just a different implementation behind the same queue interface.

Retries with Backoff and Idempotency

Even with everything local, work items fail. The model is not loaded yet, the worker thread is busy, or the browser evicted the storage cache. The fix is retries with exponential backoff and jitter — the same technique networked systems use, minus the network.

async function retry(fn, { maxAttempts = 4, baseDelay = 250 } = {}) {
  for (let attempt = 1; ; attempt++) {
    try {
      return await fn();
    } catch (err) {
      if (attempt >= maxAttempts) throw err;
      const delay = baseDelay * 2 ** (attempt - 1) + Math.random() * 100;
      await new Promise((r) => setTimeout(r, delay));
    }
  }
}

Pre-caching is the offline retry strategy that runs before the first failure. A service worker that installs the model and policy weights during the initial visit moves the expensive download to a moment when the user is actively engaged, instead of a later offline turn when a fetch would fail:

self.addEventListener("install", (event) => {
  event.waitUntil(
    caches.open("relu-weights-v1").then((cache) =>
      cache.addAll(["/models/minilm-quant.onnx", "/models/policy.json"])
    )
  );
});

The worker also lets the app answer from cached weights while a fresh version downloads in the background, so a cache miss degrades to a slow load, never a dead end.

Retries only make sense when steps are idempotent or cheaply re-executable. Embedding a query is naturally idempotent when the embedding cache is keyed by the query text — ReLU.chat memoizes query embeddings, so a retried embed is a cache hit. Retrieval is pure given the same query. Generation must be treated as non-idempotent if it has side effects, so persist its output before retrying anything downstream.

Sync-Free Design Means No Conflicts

A conventional app with a backend needs sync: the client and server both hold state, and reconciling them produces merge conflicts, last-write-wins races, and offline edit problems. A sync-free design eliminates this entire class of bugs by construction.

Sync-free means there is exactly one copy of every piece of state, and it lives on the device. There is nothing to reconcile and nothing to merge. The session-memory machinery — up to 30 turns with importance-based eviction, the recent 5 turns protected, older responses compressed to 120-character summaries, an EMA summary vector, and entity decay — is local policy. It never needs to coordinate with another copy, so it never conflicts.

The trade-off is real and should be stated plainly: no cross-device continuity. A conversation on one phone is not on the next. For a privacy-first product whose selling point is that nothing leaves the device, that trade is the point.

Key Takeaway

Offline-first is the correct default for browser-based chat: persist the work pipeline as a queue, retry with backoff and idempotent steps, and embrace a sync-free design where a single local copy of state makes conflict resolution unnecessary. The architecture that survives network failure, page reloads, and model-load hiccups is the same one that makes on-device chat fast and private.