A browser tab can stay open for hours, and a chat session inside it keeps accumulating state: query text, response DOM nodes, embeddings, entities, retrieval caches. In a server-based app this state lives on the backend and the browser holds only the visible window. In a browser-based chatbot everything lives in the page, and every turn adds a little more. Without discipline, memory grows without bound until the tab janks, the browser kills it, or the user gives up and reloads — losing the conversation.

The fix is not to avoid memory, but to bound every growing structure and give each one an eviction policy. ReLU.chat's session memory caps history at 30 turns with importance-based eviction, compresses old responses, and maintains a fixed-size summary vector — a useful template for any long-running chat page.

Where the Memory Goes

Four places dominate in a chat session:

  • DOM nodes. Streaming responses append text and math nodes. When a message is replaced or a conversation view is re-rendered, detached nodes can survive if anything still references them — an event listener, a closure, or a library that kept a handle.
  • Embedding vectors. A 384-dimensional float32 embedding is 1.5 KB. A thousand turns of query and response embeddings is 3 MB — manageable, but unbounded if the session runs all day.
  • Caches. Retrieval caches, entity stores, LRU caches, and memoized computations all grow with session length unless they are capped.
  • Closures and timers. Streaming code uses requestAnimationFrame loops and timeouts; a loop that forgets to cancel keeps the renderer and its captured objects alive.

The common thread: every one of these is a structure that grows per turn. The remedy is a per-turn budget plus an eviction rule.

Bounded History with Importance-Based Eviction

The most basic policy is a fixed-size turn buffer, and the simplest eviction is FIFO — drop the oldest turn. But FIFO is wrong for chat. An old turn that introduced entities or clarified the topic is worth more than a recent filler exchange. ReLU.chat uses a 30-turn cap with importance-based eviction, and the 5 most recent turns are always protected — they carry the immediate conversational context and should never be evicted just because an older turn scored higher.

Importance is scored from cheap features available on each stored turn:

  • whether the turn contained entities,
  • whether it was flagged as ambiguous or a topic correction,
  • how many knowledge fragments it used (fragment diversity),
  • how recently it was active.

When the buffer is full, the least-important unprotected turn is removed. In JavaScript:

function evictIfNeeded(turns, max = 30, protectedCount = 5) {
  if (turns.length <= max) return turns;

  const candidates = turns.slice(0, turns.length - protectedCount);
  candidates.sort((a, b) => importance(a) - importance(b));
  const doomed = candidates[0];

  return turns.filter(t => t.id !== doomed.id);
}

function importance(turn) {
  return (turn.hasEntities ? 2 : 0)
       + (turn.wasAmbiguous ? 1 : 0)
       + Math.min(turn.fragmentCount, 3)
       + (turn.recencyBonus || 0);
}

The numbers are illustrative; the principle is that eviction should consider what the turn contributes, not just how old it is.

Compression, Summary Vectors, and Cache Discipline

Keeping raw response text for all 30 turns is wasteful, because most of it will never be read again. ReLU.chat compresses responses to 120-character summaries once a turn is more than 5 turns old. The compression is irreversible — a deliberate trade of fidelity for longevity. The summary preserves the topic and the answer's gist, which is all the retriever and the policy need when they look backward.

Compression handles text, but a chat system also needs a compact representation of "what the conversation is about" for tasks like follow-up detection and topic continuity. ReLU.chat maintains an exponential moving average (EMA) of query embeddings:

ema = α · ema_old + (1 - α) · new_embedding

With α = 0.75, each new turn contributes 25% of the running summary, so the summary vector tracks recent drift while retaining older context. The EMA vector is fixed-size — 384 floats — no matter how long the session runs, which is exactly the kind of bounded representation you want.

Entities get the same treatment. ReLU.chat applies entity decay with a half-life of 5 turns: an entity's weight halves roughly every 5 turns after its last mention. Entities that stop appearing fade out of the working set instead of accumulating forever, and the fade is gradual rather than an abrupt deletion.

The same "bounded structure" discipline applies to the supporting caches:

  • Query embedding memoization. Repeated queries return the cached embedding instead of re-running the encoder. The memo table itself should be an LRU with a cap, not a plain object that grows forever.
  • Bounded top-k selection. Retrieval should select the top-k candidates with a partial selection algorithm, not a full sort of every fragment. ReLU.chat's retrieval does bounded top-k selection, avoiding the O(n log n) full-sort cost on every query.
  • LRU caches for retrieval results and rendered output, with explicit size limits.
  • Pre-built BM25 IDF scores. Term statistics are computed once at load, so scoring never recomputes corpus statistics per query.

The pattern is the same at every layer: any Map or array indexed by turn number or query text is a candidate leak unless it has a cap.

Detached DOM, Streaming Hygiene, and Measuring

Streaming rendering — ReLU.chat reveals responses in roughly 40-character chunks via requestAnimationFrame — is a common source of leaks if done carelessly:

  • Cancel the animation loop when the message completes, and clear the timeout on unmount.
  • When replacing a message's rendered content, remove old nodes before inserting new ones.
  • Prefer event delegation (one listener on the conversation container) over per-node listeners; per-node listeners keep every removed node alive.
  • If you attach listeners for one-off UI, remove them in a cleanup function.

A simple pattern for the streaming loop:

let rafId = null;

function streamChunks(messageEl, chunks, onDone) {
  let i = 0;
  const step = () => {
    if (i >= chunks.length) { onDone(); return; }
    messageEl.insertAdjacentText("beforeend", chunks[i++]);
    rafId = requestAnimationFrame(step);
  };
  rafId = requestAnimationFrame(step);
}

function stopStreaming() {
  if (rafId !== null) cancelAnimationFrame(rafId);
}

Before optimizing, measure. Chrome DevTools Memory panel offers heap snapshots; compare a snapshot at session start with one after a hundred turns and look at what grew. The performance.measureUserAgentSpecificMemory() API (Chromium) reports JavaScript memory attributable to the page and can be polled across a long session to confirm the curve is flat after the initial warm-up:

if ("measureUserAgentSpecificMemory" in performance) {
  performance.measureUserAgentSpecificMemory().then(r => {
    console.log(`JS memory: ${(r.bytes / 1048576).toFixed(1)} MB`);
  });
}

A healthy long session shows an initial jump (model weights, first messages) and then a flat or slowly oscillating curve. A steady linear climb means something per-turn is being retained.

Key Takeaway

Memory leaks in long chat sessions are usually not exotic — they are unbounded per-turn growth in buffers, DOM, caches, and closures. Bound every structure, give each one an eviction rule that reflects value rather than age, compress what you keep, and measure the curve over a long session. A 30-turn cap with importance-based eviction, response compression, and a fixed-size summary vector keeps a session stable for hours.