Retrieval chatbots answer questions by selecting passages from a fixed knowledge base instead of generating new text. That design has a useful side effect: every bias in the pipeline shows up in the answers, and unlike a generative model, a retrieval bot can point at the exact fragment behind each claim. Bias can enter at every stage — corpus selection, the embedding model, term weighting, ranking fusion, and entity extraction — so auditing means checking each stage separately rather than testing the chatbot as a black box.

This article walks through the sources of bias in a retrieval chatbot and gives a practical audit procedure you can run against your own system, using the architecture behind ReLU.chat (field-weighted BM25 combined with dense cosine similarity) as a concrete example.

Where Bias Enters the Pipeline

A retrieval chatbot is a chain of components, and each component can introduce or amplify bias.

Corpus selection. The largest source of bias is what was included in the knowledge base in the first place. If a base covers a topic using a small set of textbook excerpts, the answers reflect those excerpts, not the field. Coverage decisions — which topics, which authors, which eras, which dialects — become the chatbot's worldview. Metadata such as source_confidence records how much a source is trusted, but it cannot repair an entry that was never added.

Embedding model. Dense retrieval embeds queries and fragments with a sentence transformer. ReLU.chat uses all-MiniLM-L6-v2, quantized to ONNX and run in-browser; the model was trained on web-scale text, so its 384-dimensional vectors inherit associations from that text. Occupational gender skews, regional phrasing preferences, and topic correlations from the training corpus all live in the embedding space, and they do not disappear just because the knowledge base itself is clean.

Term weighting. Sparse retrieval weights terms by frequency. BM25's idf term gives rare terms more weight, which is usually desirable. But indexing decisions change the math: if entry names are repeated three times and aliases twice during indexing, as ReLU.chat does to emphasize field relevance, the repetition shifts both term frequencies and idf. Terms that appear in many popular entries can end up systematically over- or under-weighted.

Ranking and fusion. When sparse and dense scores are combined with fixed weights, the blend decides whose opinion wins on borderline cases. ReLU.chat fuses 70% dense and 30% sparse scores. If the dense model favors fluent generic phrasing while BM25 favors exact terminology, that weight split determines which fragments reach the top for ambiguous queries — and users mostly read the top fragments.

Entity extraction. Regex and fuzzy matching map mentions to entities. A three-pass extractor (exact alias regex, then fuzzy word-overlap with Levenshtein and substring containment, then notation pattern matching) can mis-map names: a common name can shadow a less common one, and fuzzy matching can collapse distinct entities that share words. Every mis-mapping is a small bias that propagates into intent classification and retrieval.

Auditing the Corpus

Corpus audits answer one question: what could this chatbot never say?

Coverage profiling. List the entities, topics, and question types the knowledge base supports, then generate probe questions for each. For every probe, record whether a relevant fragment exists at all. Missing fragments are not a ranking bug; they are a coverage gap, and no retrieval tuning fixes them.

Provenance review. Record where each entry came from and how confident the source is. ReLU.chat entries carry truth_confidence and source_confidence metadata; a review pass that flags low-confidence entries for verification is an audit in itself. A corpus that draws 80% of its entries from one author has a perspective problem even if every fact is individually correct.

Perspective diversity. For contested topics, check whether multiple viewpoints are represented. Bias in a retrieval chatbot often shows up as single-answer monotony: every phrasing of the question returns fragments from the same source, because that source dominates the embedding neighborhood and the term statistics.

Auditing Retrieval Behavior

Corpus audits tell you what could be retrieved; retrieval audits tell you what actually gets retrieved.

Build probe sets. Create query sets that vary one dimension at a time: gendered versus neutral phrasing, regional variants, formal versus informal wording, and exact-term versus paraphrase versions of the same question. Run each set through the retriever and inspect the top-k fragments.

Check rank stability. A well-behaved retriever returns similar top-k sets for paraphrases of the same question. Large divergence on wording alone is a red flag — it usually means the query embedding or term matching latched onto a spurious word such as a rarely used synonym.

Look at position bias. Users almost always read the first one or two fragments. If a biased fragment consistently ranks first while a neutral alternative ranks fourth, the system has a practical bias even though both fragments exist. Position bias is the most common way a "neutral" knowledge base produces biased answers.

Auditing Answers End to End

Retrieval metrics measure relevance; bias audits measure fairness across groups and framings.

Disaggregated evaluation. Score answers on a labeled set, then split the results by query framing: gender, dialect, formality. A system with good overall accuracy can still show large per-group gaps. Report the gaps, not just the average, and treat a large gap on any group as a defect even when overall numbers look fine.

Traceability. Because retrieval answers come from fragments, every answer should be traceable to the fragment that produced it. ReLU.chat's policy decides how many fragments to use per response via its frag_count action head; auditing means checking that the chosen fragments actually support the claims in the answer. Generative chatbots cannot offer this audit. Retrieval chatbots can, and should — make the trace part of the review workflow.

Log and review. Record queries, top-k fragments, and final answers, then sample the log for mis-retrievals. A lightweight review loop run after every content update catches regressions that aggregate metrics hide.

Key Takeaway

Bias in retrieval chatbots is not one defect; it enters through corpus selection, embedding models, term weighting, fusion weights, and entity extraction. Audit each stage separately: profile coverage, probe retrieval with controlled query sets, and evaluate answers with disaggregated metrics. The payoff of a retrieval architecture is that every answer can be traced to a source fragment — which turns bias auditing from guesswork into an inspectable, repeatable process.