Users type short, sloppy queries. "neural net" when the knowledge base says "neural network." "bayes therom" when the topic is Bayes' theorem. A lexical retriever such as BM25 matches terms exactly, so every misspelling and every synonym costs recall. The common modern answer is to bolt on an LLM that rewrites the query — expensive, slow, and for on-device products, often impossible.

Classic information retrieval solved this problem decades ago without generative models. Query expansion — adding related terms to a query before retrieval — plus spelling correction and term boosting covers most of the vocabulary-mismatch gap. This post shows how those techniques work, using the retrieval stack of ReLU.chat as a concrete reference.

The Vocabulary Mismatch Problem

BM25 scores a document by the terms it shares with the query. If the query uses a word the document never uses, that term contributes zero. Two sentences can mean the same thing and share no vocabulary: "What is the prisoner's dilemma?" versus "Game theory's classic two-player problem." Dense embeddings solve paraphrase matching but have their own failure modes — they blur proper nouns and aliases, and they need a 22 MB model loaded to run at all.

The practical answer is to attack the mismatch on both sides of the pipeline: enrich the index with aliases and field weights, and enrich the query with synonyms, corrections, and boosts. Neither needs a language model.

Expand at Index Time with Field Weights

Indexing is the cheapest place to buy recall. When a knowledge base entry has a name and several aliases, index all of them — with weights that say which is the canonical form. ReLU.chat does exactly this: entry names are repeated 3x and aliases 2x during indexing, into a field-weighted BM25 with k1 = 1.5 and b = 0.75. Repeating a name triples its term frequency contribution, which is a simple, invertible way to boost exact matches without distorting the whole corpus. Bigram phrase matching adds another recall channel: "neural network" is also indexed as the phrase pair, so partial overlaps still score.

def bigrams(terms):
    return [f"{a} {b}" for a, b in zip(terms, terms[1:])]

Index-time expansion has a big advantage: the query stays untouched, so there is no risk of corrupting the user's intent. The cost is index size, which for a curated knowledge base is trivial.

Expand at Query Time with Synonyms

Query-side expansion works best for closed vocabularies where you control the entities. A three-pass entity extraction pass — exact alias regex, fuzzy word-overlap with Levenshtein distance and substring containment, and notation pattern matching — finds the entities a query refers to. Once the entity is identified, its canonical name and aliases can be appended to the query with reduced weight:

def expand(query_terms, alias_map):
    for term in query_terms:
        yield term, 1.0
        for alias in alias_map.get(term, []):
            yield alias, 0.5  # expansion terms count half as much

The reduced weight matters. An expansion term is a guess, not the user's intent, so it should boost documents containing it without letting it dominate. This is the same principle as the index-side 3x/2x weights, applied on the other side of the scoring equation.

Fix Spelling with Edit Distance

Spelling correction without an LLM is edit-distance candidate generation over a vocabulary. For each query token, generate candidates within a small Levenshtein distance, score them, and if the best candidate beats the original token convincingly, substitute it.

import Levenshtein

def correct(token, vocab, max_dist=2):
    best, best_dist = token, max_dist + 1
    for word in vocab:
        d = Levenshtein.distance(token.lower(), word)
        if d < best_dist:
            best, best_dist = word, d
    return best if best_dist <= max_dist else token

Two rules keep this safe. First, never auto-correct proper nouns and entity names — that is what the alias pass is for, and entity extraction already handles fuzzy matching with Levenshtein and substring containment. Second, do not correct tokens below a minimum length or above a distance threshold; "bayes" to "Bayes" is a win, but "game" to "gaze" is a disaster.

Boost What Matters, Then Re-Rank

Expansion improves recall; ranking restores precision. The retrieval stage should return a generous candidate set, and a second stage should re-rank it with the features that matter most.

ReLU.chat's design is an ensemble: dense cosine similarity over MiniLM embeddings and sparse field-weighted BM25, with rank fusion at 70% dense / 30% sparse. Expansion feeds both sides — synonyms help the sparse side directly, and the entity/alias mapping helps the dense side by making the query vector align with the canonical form of the topic. The weights (70/30) are not arbitrary; they encode the observation that paraphrase matching is usually stronger than lexical matching, while lexical matching is indispensable for exact names and aliases.

One caution: expansion amplifies noise as well as recall. A query that matches no entity and no alias should pass through untouched, because injecting arbitrary synonyms into an already ambiguous query retrieves fragments about the wrong topic entirely. The entity-extraction pass is the gate that decides whether expansion applies — no entity, no expansion. This keeps the pipeline deterministic and testable: the same query always produces the same expanded query, which is exactly the property an offline evaluation set needs.

The pipeline shape — expand, retrieve broadly, re-rank — is the same one LLM-based systems use, minus the LLM.

Key Takeaway

Query expansion without an LLM is a combination of three cheap, deterministic techniques: enrich the index with repeated names and aliases (3x names, 2x aliases, bigram phrases), expand the query with lower-weighted synonyms from entity extraction, and correct spelling with edit distance while protecting proper nouns. Retrieval quality comes from the pairing of broad recall and a 70/30 dense-sparse re-ranking stage — no generative model required.