BM25 (Best Matching 25) is the workhorse of sparse retrieval: a ranking function that scores documents by how well they match a query's terms, without any neural machinery. Dense retrieval gets the headlines, but BM25 still anchors most hybrid systems because it is fast, robust, and needs no training. ReLU.chat runs field-weighted BM25 (with k1 = 1.5, b = 0.75) alongside dense cosine similarity, fusing the two rankings in a 70% dense / 30% sparse ensemble. To understand why that blend works, it helps to actually compute a BM25 score by hand.

The Formula

BM25 scores a document D against a query Q as a sum over the query's terms:

score(D, Q) = Σ_t  IDF(t) · [ f(t, D) · (k1 + 1) ] / [ f(t, D) + k1 · (1 - b + b · |D| / avgdl) ]

The pieces:

  • f(t, D) — the term frequency of term t in document D.
  • |D| — the length of D in tokens.
  • avgdl — the average document length across the corpus.
  • k1 — a term-frequency saturation parameter, typically 1.2–2.0. ReLU.chat uses 1.5.
  • b — a length-normalization strength from 0 to 1. ReLU.chat uses 0.75.

The IDF (inverse document frequency) term uses smoothed log:

IDF(t) = ln( 1 + (N - n(t) + 0.5) / (n(t) + 0.5) )

where N is the number of documents and n(t) is the number of documents containing t. The +0.5 smoothing prevents division by zero and keeps IDF positive for terms that appear in every document (where the raw formula would give zero or negative values).

A Worked Example

Consider a three-document corpus:

Doc1: the prisoner's dilemma is a classic game theory example
Doc2: game theory studies strategic decision making
Doc3: a dilemma is a situation requiring a choice not a game

Token counts: |Doc1| = 9, |Doc2| = 6, |Doc3| = 11, so avgdl = (9 + 6 + 11) / 3 = 8.67.

Query: game theory dilemma.

Document frequencies: game appears in all 3 documents, theory in Doc1 and Doc2, dilemma in Doc1 and Doc3.

IDF(game)    = ln(1 + (3 - 3 + 0.5) / (3 + 0.5)) = ln(1.143) = 0.134
IDF(theory)  = ln(1 + (3 - 2 + 0.5) / (2 + 0.5)) = ln(1.600) = 0.470
IDF(dilemma) = ln(1 + (3 - 2 + 0.5) / (2 + 0.5)) = ln(1.600) = 0.470

Now the term-frequency factor for each document. For Doc1 (|D| = 9), the denominator for every query term is the same:

1 - b + b · |D| / avgdl = 1 - 0.75 + 0.75 · 9 / 8.67 = 1.029
k1 · 1.029 = 1.543

With f = 1 for each of the three terms:

tf-factor = 1 · (1.5 + 1) / (1 + 1.543) = 2.5 / 2.543 = 0.983

Doc1 score:

game    : 0.134 · 0.983 = 0.131
theory  : 0.470 · 0.983 = 0.462
dilemma : 0.470 · 0.983 = 0.462
total   : 1.055

Doc2 (|D| = 6, game and theory present, no dilemma):

1 - b + b · |D| / avgdl = 0.25 + 0.75 · 6 / 8.67 = 0.769
tf-factor = 2.5 / (1 + 1.5 · 0.769) = 2.5 / 2.154 = 1.161
game   : 0.134 · 1.161 = 0.155
theory : 0.470 · 1.161 = 0.545
total  : 0.700

Doc3 (|D| = 11, game and dilemma present, no theory):

1 - b + b · |D| / avgdl = 0.25 + 0.75 · 11 / 8.67 = 1.202
tf-factor = 2.5 / (1 + 1.5 · 1.202) = 2.5 / 2.803 = 0.892
game    : 0.134 · 0.892 = 0.119
dilemma : 0.470 · 0.892 = 0.419
total   : 0.538

Final ranking: Doc1 (1.055) > Doc2 (0.700) > Doc3 (0.538). Doc1 wins because it contains all three query terms, and the length penalty it pays for being longer than Doc2 is outweighed by covering the full query.

What the Parameters Do

k1 controls term-frequency saturation. With a small k1, the first occurrence of a term earns most of the available credit and further occurrences add little; with a large k1, term frequency keeps mattering longer. In the limit k1 → ∞ the factor approaches the raw term frequency. ReLU.chat's k1 = 1.5 is a typical middle ground: repeated terms help, but a document does not win by stuffing one word.

b controls length normalization. b = 0 removes length normalization entirely (a long document is not penalized for being long), while b = 1 applies full normalization. ReLU.chat's b = 0.75 is a fairly strong length penalty, which suits a retrieval corpus with short knowledge fragments where length correlates with topic dilution. Notice in the example how Doc2, at 6 tokens, gets a larger tf-factor (1.161) than Doc3 at 11 tokens (0.892) purely because it is shorter.

The smoothed IDF makes rare terms matter more, but it also keeps a term appearing in all documents from contributing zero — a common trap in naive log(N/n) implementations.

Field Weighting, Phrase Matching, and Hybrid Fusion

In a structured corpus, terms in important fields should count more. One clean way to implement field weighting without a multi-field formula is to boost term frequencies at index time. ReLU.chat repeats entry names 3 times and aliases 2 times when building the BM25 index — which is mathematically equivalent to multiplying that term's frequency by 3 or 2 in the tf-factor. The effect is that a match on a canonical entry name contributes roughly triple the weight of a match in a description field, while the ranking formula itself stays unchanged.

Phrase matching works similarly: the index also stores bigrams from the entries, so a query like game theory matches the indexed bigram game theory in addition to the individual terms. This catches phrases whose words are individually common but meaningful together.

A compact Python implementation of the score:

import math

def bm25_score(query_tokens, doc_tokens, idf, avgdl, k1=1.5, b=0.75):
    score = 0.0
    dl = len(doc_tokens)
    for t in query_tokens:
        tf = doc_tokens.count(t)
        if tf == 0:
            continue
        denom = tf + k1 * (1 - b + b * dl / avgdl)
        score += idf.get(t, 0.0) * (tf * (k1 + 1)) / denom
    return score

The idf dict is precomputed once per corpus (ReLU.chat pre-builds BM25 IDF at load time) so query-time scoring is just a loop over query terms.

Finally, remember that BM25 scores are only meaningful within a single query — the absolute values depend on the query and corpus statistics, so you rank by score rather than thresholding it naively. In hybrid retrieval, the sparse score and the dense cosine similarity live on different scales, so they are normalized and fused by rank rather than by raw value. ReLU.chat combines the two with a 70% dense / 30% sparse weight on the fused rank. The sparse leg contributes lexical precision — exact terms and names — while the dense leg contributes semantic recall, and the fixed fusion weight keeps the behavior predictable.

Key Takeaway

BM25's score is the product of three intuitions: rare terms matter more (IDF), term frequency saturates (k1), and longer documents should not win just by being long (b). The worked example shows the mechanics: a document containing all query terms outranks shorter partial matches, and the length penalty is mild enough not to override full coverage. Field weighting via term repetition and bigram phrase matching extend the same formula to structured corpora without changing a line of the math.