Blog
Technical articles about on-device AI, browser-based chatbots, NLP, reinforcement learning, and privacy-first machine learning.
Measuring Retrieval Quality: Precision, Recall, MRR and NDCG
Retrieval quality is measured with four standard metrics: precision and recall for top-k usefulness, MRR for how early the first good answer appears, and NDCG for graded relevance with position discounting — tuned against a small labeled eval set.
Read article →
Chunking Strategies for Knowledge Bases: Splitting Text for Retrieval
Chunk size, boundaries, and overlap decide what a knowledge base retriever can find; structure-aware chunks serve hybrid BM25-plus-dense retrieval best.
Query Expansion Without an LLM: Synonyms, Spelling and Boosting
Query expansion without an LLM combines index-time alias weighting, lower-weighted query-side synonyms from entity extraction, edit-distance spelling correction that protects proper nouns, and a dense/sparse re-ranking stage.
How Knowledge Bases Are Structured: Fragments, Fields and Metadata
Knowledge bases work best when entries are split into typed fragments, names and aliases are field-weighted for BM25, and metadata steers ranking and policy.
IndexedDB for On-Device AI: Storing Embeddings and Session State Locally
IndexedDB stores embeddings as raw binary and session state as structured records, giving browser-based AI a complete local persistence story that never leaves the device.
Float32 vs Float64: Precision Trade-Offs for In-Browser ML
JavaScript numbers are float64 but ML tensors default to float32. Use float32 for model I/O and similarity scoring, float64 for accumulators and iterative updates where error compounds, and int8 quantization to shrink weights in memory.
The Real Cost of Cosine Similarity: Optimizing Vector Math in JavaScript
Cosine similarity looks cheap but is O(N) per query in the browser; precomputed norms, typed arrays, bounded top-k, and memoization cut the real cost.
Offline-First Chat Architecture: Queues, Retries and Sync-Free Design
Offline-first is the natural architecture for browser-based chat: persist the turn pipeline as a queue, retry with backoff and idempotent steps, and embrace sync-free design where a single local copy of state eliminates conflict resolution entirely.
Model Size vs Quality: Why a Small Distilled Model Can Beat a Big One on Edge Devices
Quality is task-relative. A small distilled model fine-tuned for a narrow task — quantized, cache-local, paired with strong retrieval — can beat a much larger general model on edge devices.
Conversation State Machines: Modes, Intents and Follow-Up Routing
Modes, intents, and follow-up routing form a conversation state machine: persistent states, per-turn actions, and a routing table that ties them together.
Debugging In-Browser ML: Profiling, Logging and Graceful Fallbacks
Debugging ML that runs in the browser needs three layers of defense: profiling each inference stage, logging structured on-device events, and building a graceful fallback ladder so a model failure degrades to a slower answer instead of a blank page.
Bias in Retrieval Chatbots: Where It Comes From and How to Audit It
Bias enters retrieval chatbots through the corpus, embeddings, term weighting, ranking fusion, and entity extraction; this article shows how to audit each stage.
Privacy Checklist for Browser-Based AI: What Actually Leaves Your Device
A six-step audit for browser-based AI: inventory first-load requests, watch live conversations, test offline, inspect storage, and read the source.
Named Entity Recognition Without an LLM: Regex, Fuzzy Matching, and Knowledge Bases
For closed domains, NER does not need an LLM: exact alias regex, fuzzy matching with Levenshtein, notation patterns, and a knowledge base extract entities in milliseconds, on-device.
The Math of BM25: A Worked Example of Sparse Retrieval
A hand-computed BM25 ranking example showing smoothed IDF, term-frequency saturation, length normalization, and field weighting via term repetition.
WebAssembly and On-Device ML: Why WASM Makes Browser AI Fast
WebAssembly lets C and Rust numeric kernels run at near-native speed in the browser, and it is the substrate under on-device ML runtimes like ONNX Runtime and transformers.js.
Latency Budgets for Conversational UI: The 400ms Response Rule
Users experience latency as time to first token, not time to completion. Budget 400 ms for that first token on your slowest device, and stream everything after it.
KaTeX vs MathJax: Rendering Math in the Browser
A practical comparison of KaTeX and MathJax for in-browser math rendering: speed, bundle size, TeX coverage, accessibility, and streaming output.
INT8 Quantization Explained: From FP32 to 8-Bit Without Losing Accuracy
INT8 quantization maps a model's weights and activations onto 256 integer buckets, cutting memory 4x while usually keeping accuracy — the key to fitting models in a browser.
Cosine Similarity and Vector Search: The Math Behind Semantic Retrieval
Cosine similarity measures the angle between embeddings, and with normalized vectors it becomes a tight dot-product loop that powers semantic retrieval entirely in the browser.
Memory Management in Long Browser Chat Sessions: Avoiding Leaks
How bounded turn buffers, importance-based eviction, response compression, and cache discipline keep long browser chat sessions from leaking memory.
Service Worker Caching Strategies for ML Assets
Model weights are immutable and large, which makes them ideal cache-first assets. A practical guide to service worker strategies for ML: precache, versioning, and quota.
Intent Classification with Prototype Embeddings: Zero-Shot Style Without an LLM
Prototype embeddings classify intent with cosine similarity against reference vectors — no LLM, no training data, and no server round-trip.
Subword Tokenization for On-Device NLP: How BPE Works
Byte Pair Encoding builds a compact subword vocabulary by merging the most frequent character pairs, and it is why small on-device models can handle full natural language without giant embedding tables.
Web Workers: Keeping the UI Thread Responsive During Inference
Inference is compute, and compute does not belong on the UI thread. Here is how dedicated workers, transferable buffers, and worker pools keep a browser-based AI chatbot responsive.
Accessibility in AI Chatbots: WCAG Compliance for Conversational UI
How to make browser-based chatbots accessible: ARIA landmarks, keyboard navigation, reduced motion, and contrast compliance.
Progressive Loading: BOW to MLP to Transformer in 3 Stages
The three-stage loading strategy that gives usable answers instantly while the 22MB transformer model streams in.
Dense-Sparse Ensemble Ranking: Combining BM25 and Embeddings
How fusing dense cosine similarity with BM25 sparse scores at 70/30 weights produces better retrieval than either alone.
Topic Correction: When Users Say 'No, I Meant X'
How explicit topic correction detection prevents chatbot topic drift when users redirect the conversation mid-turn.
Follow-Up Detection: 20 Patterns That Make Chatbots Conversational
How regex pattern matching and heuristic detection handle 20+ follow-up types from 'how?' to 'tell me more' to emoji-only responses.
BM25 in the Browser: Field-Weighted Sparse Retrieval Without a Server
Implementing field-weighted BM25 with bigram phrase matching entirely in JavaScript for real-time knowledge retrieval.
Service Worker Model Pre-Caching: Zero-Wait Chatbot Loading
How service workers pre-cache the 22MB transformer model and policy weights for instant chatbot startup.
Session Memory Architecture: 30 Turns of Context Without a Server
How importance-based eviction, response compression, and EMA summary vectors maintain long conversation context entirely in the browser.
Progressive Streaming Rendering: Making Chatbots Feel Native
How progressive response rendering with requestAnimationFrame creates a natural typing effect in browser chatbots.
Int8 Quantization in the Browser: How We Made MLP Inference 4x Faster
How symmetric int8 quantization reduces memory and speeds up policy network inference in pure JavaScript.
ONNX Runtime Web and INT8 Quantization: How 90MB Models Become 22MB
A practical look at how ONNX Runtime Web plus INT8 post-training quantization shrinks a 90MB sentence transformer to roughly 22MB, with only a small accuracy loss, and why that tradeoff is exactly right for in-browser NLP.
BM25 vs Dense Retrieval: Why We Use Both for Browser Chatbots
BM25 is fast and exact; dense embeddings are semantic and forgiving. For real user queries — many of them short, typo-prone, or relying on jargon — neither wins alone. Here is why hybrid retrieval is the right default.
Service Workers and PWA Caching for Offline AI Chatbots
A service worker can pre-cache the 22MB ONNX model, the embedding index, and the chatbot shell so the entire experience keeps working after the first load — even on a plane.
WebGPU and the Future of In-Browser AI Inference
WebGPU has shipped in stable Chrome, Edge, Firefox, and Safari. For in-browser ML, it is the unlock that makes 7B-parameter models possible without a server. Here is what it actually changes — and what it does not.
Knowledge Fragment Composition: Why We Ditched Generation for Retrieval
Generation is flexible and dangerous; retrieval is rigid and safe. For domain-specific chatbots where accuracy matters more than creativity, retrieval + fragment composition wins. Here is the design and the tradeoffs.
Sentence Embeddings Explained: Why 384 Dimensions and What They Mean
384 is a specific number with a specific history. It is the output dimension of MiniLM-L6-v2, a distilled sentence transformer. Here is what those dimensions actually encode, why the number is what it is, and why it is the right size for browser NLP.
PPO and Reward Design: Training a Chatbot Policy Network That Actually Works
The hardest part of training a chatbot policy network is not the algorithm — it is the reward function. A bad reward produces a model that optimizes for the wrong thing. Here is the function we use, what we tried that did not work, and what we learned about the dark art of reward shaping.
Heuristic Fallback for Cold Start: Graceful Degradation in Browser AI
A 22MB model takes seconds to load, and users will not wait. A heuristic fallback gives a working chatbot on first paint and hot-swaps to the full transformer when the model finishes downloading. Here is how the pattern works and what it costs.
Linguistic Connectors: The Small Detail That Makes Retrieval Bots Sound Human
A retrieval chatbot that just concatenates fragments sounds robotic. Connectors — small transitional phrases — turn a list of facts into a coherent response. Here is the small but important design decision that affects how a knowledge-grounded bot feels to talk to.
Sentence Embeddings vs Word Embeddings: Why We Embed Whole Questions
Word embeddings (Word2Vec, GloVe) capture the meaning of individual words. Sentence transformers (MiniLM, SBERT) capture the meaning of a whole phrase. For Q&A retrieval, only the second is useful. Here is why averaging word vectors fails, and what sentence-level embeddings do differently.
Privacy-First AI: Why Browser-Based Machine Learning Matters
An argument for why browser-based machine learning is the future of privacy-first AI, and how ReLU.chat achieves zero data collection.
Training Chatbots with Reinforcement Learning: The ReLU.chat Approach
An exploration of how reinforcement learning trains a lightweight MLP policy network to compose chatbot responses from knowledge fragments, without any LLM dependency.
How Browser-Based Chatbots Work: A Deep Dive into On-Device NLP
A technical deep dive into how modern browser-based chatbots run full NLP pipelines client-side using quantized ONNX models, BM25 retrieval, and RL-trained policy networks.