How ReLU.chat Works

1. Overview

Questions and answers are processed on your device. The website still downloads page assets and, when you choose enhanced matching, a quantized MiniLM model of about 22 MB. No language-generation API is used.

1. Start: load the selected knowledge base and build its keyword index.
2. Retrieve: combine lexical intent cues, entity matches, BM25 scores, and available vector similarities.
3. Decide: run the 13,079-parameter policy in verified WebAssembly or JavaScript; use built-in rules if assets are unavailable.
4. Explain: render the complete answer once, with mathematical notation and available source links.

The optional embedding upgrade prepares every vector before switching the active backend. Cached query vectors are cleared at the switch. Public knowledge embeddings are cached in IndexedDB under a hash of the model, knowledge content, and intent examples. Conversations remain in memory unless you save a text copy.

Browse all six chatbots · Inspect the measured policy results

2. Signal Layer

After embedding, a lightweight signal layer prepares a structured DecisionPacket for the policy network. It combines multiple retrieval and classification signals into a coherent pre-policy feature bundle.

1. BM25 Sparse Retrieval — Field-weighted term-frequency scoring (k1=1.5, b=0.75). Entry names are repeated 3× and aliases 2× during indexing, so name-matched terms naturally score higher via TF boosting. Bigram phrase matching catches multi-word queries like "Nash equilibrium" as a single token (nash_equilibrium), with an IDF-weighted bonus that complements unigram scoring.
2. Entity Extraction — Three-pass extraction: (1) exact alias regex matching, (2) fuzzy word-overlap scoring using Levenshtein distance and substring containment for typo tolerance, (3) notation pattern matching for game-theory expressions like (D,D) and (C,C). Session context enriches entities from recent turns with decay weighting.
3. Intent Classification — Cosine similarity against 19 intent prototypes per category (definition, example, formal, application, comparison), scored with a 70/30 blend of best-match and weighted-average to prevent one lucky prototype from dominating. Calibrated with temperature=1.5 softmax for reliable confidence estimates.
4. Ensemble Ranking — Dense cosine similarity (0.7 weight) and BM25 scores (0.3 weight) are fused into a combined ranking. A neural reranking pass applies a token-overlap bonus to refine the top results. Follow-up queries get a 0.35 boost (scaled by conversation depth) to the previous topic, preventing topic drift.
5. Feature Extraction — The ensemble ranking, calibrated intent scores, entity data, and session context are compiled into the 25-feature vector that feeds the policy network.

Topic Correction — The signal layer detects explicit topic corrections (e.g., "I meant X" or "no, just X") via regex pattern matching against the current query and session history. When a correction is detected, the corrected topic embedding is forced to the top of the dense-sparse ranking, overriding the ensemble score. This prevents topic drift when the user redirects the conversation mid-turn.

The signal layer is stateless and runs entirely in the browser — no server calls, no external inference APIs. The resulting DecisionPacket contains the query embedding, entity list, calibrated intent distribution, dense and sparse rankings, confidence metrics, and session context.

3. Feature Extraction

Every query produces a 25-feature vector that feeds the policy network. These features capture similarity, entity presence, intent distribution, session history, and fragment metadata.

25-Feature Layout

4. Policy Network

The policy uses 25 runtime features, two linear layers with ReLU activations, and six output heads. There is no LayerNorm in either training or inference. Training-time feature scaling is folded into the first layer on export.

Input: 25 raw features
Hidden: 25 → 128 ReLU → 64 ReLU
Heads: mode (5), intent (5), topic count (4), fragment count (4), creativity (1), tone (4)
Total: 13,079 parameters
WASM: 3,685 bytes; fixed 128 KiB memory; no host imports
Release: 2.0.0 — September 9, 2026

JavaScript inference is ready when weights load. The runtime verifies the optional WASM hash before attaching it to the same MLP. The WASM kernel produces logits; JavaScript validates the final answer plan. Failed downloads or incompatible schemas keep the basic heuristic available. Explicit greeting and help commands do not get converted into unrelated knowledge topics.

5. Training and evaluation

The CPU-only NumPy trainer uses authored examples and the actual JavaScript feature extractor. One question is one episode: this is a contextual-bandit routing task, not long-horizon conversation training. A supervised warm start is followed by on-policy REINFORCE with an action-independent reward baseline.

The September run used 1,830 training, 420 validation, and 438 held-out test cases. Topics are separated across all splits; training templates differ from the validation/test templates. Joint mode-and-intent accuracy was 93.6% for the old model, 98.6% for the supervised checkpoint, and 98.6% after REINFORCE. The complete trainer run, including export and evaluation, took 3.10 seconds on the development laptop.

These measurements describe routing on authored cases. They do not establish factual accuracy, learning outcomes, or performance on every device. A wall-clock deadline, release gate, and numerical parity checks bound the training and export process. Read the reproducible training guide.

6. Fragment-Based Response Composition

Each knowledge-base entry contains categorized fragments (def, int, ex, form, app) with metadata fields: truth_confidence, source_confidence, difficulty, style, avoid_with.

The policy produces an AnswerPlan specifying:

• Mode — normal, comparison, greeting, help, off_topic
• Intent — definition, example, formal, application, comparison
• Topics — which KB entries to include
• Fragment Plan — which categories and indices per topic
• Template — opener/closer indices, comparison opener key, connector keys
• Creativity — scalar [0,1] controlling response variation
• Tone — neutral, formal, intuitive, playful

composeV2() in core/nlp.js reads the AnswerPlan and assembles the final text by selecting fragments, applying linguistic connectors ("For instance,", "More formally,", etc.), prefixed by openers and suffixed by closers — all indexed from the plan with modulo-safety.

Immediate answer rendering

Responses are composed locally and inserted once. There is no artificial typing delay or repeated partial-HTML update. The chat log announces completed messages, and KaTeX renders mathematical notation after insertion.

Session Memory

The SessionMemory class (core/session.js) tracks up to 30 turns with importance-based eviction (recent 5 turns are always protected). Old responses are compressed to 120-char summaries after 5 turns. An EMA summary vector (exponential moving average, α=0.75) of query embeddings provides dense multi-turn context to the policy without additional feature extraction. Entity mentions decay with a half-life of 5 turns, and fragment diversity penalties prevent repetitive responses across long conversations.

Comparison Mode

When mode === 'comparison', the policy selects a comparisonOpenerKey (both, contrast, or similarity) from the template. The renderer uses patterned openers like "Both A and B are important concepts here." and distributes categories across multiple topics.

7. Action Schema & Validation

Every AnswerPlan passes through validatePlan() (policy/action-schema.js) which enforces:

• Type checking on all 10 top-level fields
• Enum membership (mode, intent, tone)
• Range validation (creativity ∈ [0,1], topics ≤ maxTopics)
• Cross-field consistency (fragmentPlan length matches topics; comparison with <2 topics falls back to normal)
• Sanitization with schema defaults for missing/invalid fields

8. Feature Serialization

For the WASM boundary, features are packed into a 107-byte buffer:

packFeatures(features) → {
  float32: Float32Array(25),     // offset 0,  100 bytes
  uint8:   Uint8Array(7),       // offset 100, 7 bytes
  buffer:  ArrayBuffer(107)      // total
}

Uint8Array layout:
  [0] = entityCount         (u8, 0-3)
  [1] = packed booleans     (bits: entityBoostHit|hasComparisonCue|hasFormalCue|hasExampleCue|wasAmbiguous)
  [2] = lastTopicAge        (u8, 0-8)
  [3] = queryLenTokens      (u8, 1-32)
  [4] = followUpType        (u8, 0-22)
  [5] = minDifficulty       (u8, 0-4)
  [6] = fragDiversity       (u8, 0-5)

9. Heuristic Fallback

When the MLP policy engine is unavailable (e.g., during cold start or weight load failure), planAnswerHeuristic() generates the same AnswerPlan structure using 15 parameterized decision thresholds covering greeting detection, off-topic handling, comparison fallback, entity boost, and creativity defaults. This ensures the system is always functional even without trained weights.

10. Open Source

The full codebase is available at github.com/yunusemrejr/relu-chat under the MIT license. This includes:

• core/ — NLP engine, chatbot engine, session memory, BM25 scorer, signal layer
• policy/ — Feature extractor, MLP inference, action schema, policy runtime
• dev/scripts/ — PyTorch training, weight export, prompt augmentation

Premium · From the makers of ReLU.chat

This page is the map. The toolkit is the vehicle.

The On-Device Chatbot Builder Toolkit turns the architecture above into a buildable system for your own domain: frameworks, templates, and evaluation guides for every stage — signals, retrieval, evidence, decision policy, composition, validation. Ship a privacy-first chatbot that runs entirely on your users' devices.

Input → Signals → Retrieval → Evidence → Decision → Composition → Validation → Response Get the Toolkit · $29 →