ReLU.chat uses two retrieval signals: dense cosine similarity (from MiniLM embeddings) and sparse BM25 scores. Neither alone is sufficient. This article explains how we fuse them.
The Two Signals
Dense similarity: The query is embedded into a 384-dimensional vector. Each knowledge base entry has a precomputed embedding. Cosine similarity captures semantic meaning — "optimal strategy" matches "best response" even without shared words.
Sparse BM25: Term-frequency scoring with IDF weighting. Captures exact lexical matches — "Nash equilibrium" matches entries containing those exact words.
The Ensemble Formula
const blended = 0.7 denseScore + 0.3 bm25Score;
The 70/30 split was determined empirically across all three chatbot domains (game theory, data science, Islamic Golden Age). Dense similarity is more robust to paraphrasing; BM25 is more precise for exact terms.
Neural Reranking
After ensemble ranking, a lightweight neural reranker applies a token-overlap bonus:
const rerankScore = (1 - blend) baseScore + blend sim01;
This refines the top-10 results by penalizing entries that share few tokens with the query.
Follow-Up Boosting
When a follow-up is detected, the previous topic gets a +0.35 boost (scaled by conversation depth). This prevents topic drift during elaboration:
const boostAmount = 0.35 (1 + conversationDepth 0.15);
Performance
For a 100-entry knowledge base:
- Dense ranking: ~0.5ms (cosine similarity × 100)
- BM25 scoring: ~0.1ms (inverted index lookup)
- Ensemble fusion: ~0.05ms (weighted sum)
- Total: ~0.65ms per query
Key Takeaway
Dense-sparse ensemble ranking gives you the best of both worlds: semantic understanding from embeddings and lexical precision from BM25. The 70/30 split works well across domains.