BM25

Overview

BM25 is a classic lexical retrieval scoring function used in search systems to rank documents based on how well their terms match a query. It comes from the probabilistic information retrieval tradition and is widely used in keyword search engines because it balances term frequency, document length, and term rarity in a practical, effective way.

For AI Product Managers, BM25 matters because it remains a foundational ranking signal even in systems increasingly shaped by embeddings, rerankers, and LLMs. In hybrid search, BM25 often provides the lexical relevance layer that complements semantic retrieval. Understanding how BM25 behaves, how its scores are interpreted, and how they can be calibrated is useful when designing search quality metrics, blending retrieval methods, or explaining ranking behavior to stakeholders.

Key Developments

• 2026-03-07: Doug Turnbull's piece Can BM25 be a probability? explored whether BM25 scores can be framed as odds versus probabilities, introducing a Bayesian interpretation. The discussion highlighted why score interpretation matters when calibrating hybrid search systems that combine lexical and probabilistic signals.
• 2026-03-10: Doug Turnbull's note Ugly hack to force BM25 to 0-1 described a pragmatic normalization technique for mapping BM25 scores into a 0–1 range. This reflected a practical product and engineering need: making lexical retrieval scores easier to compare with other model outputs in production systems.

Relevance to AI PMs

• Designing hybrid retrieval systems: BM25 is often paired with vector search in RAG and search products. PMs need to understand its role so they can make better decisions about blending lexical and semantic retrieval, especially for exact-match, acronym, and rare-term queries.
• Score calibration and ranking explainability: BM25 scores are not naturally intuitive to non-specialists. Knowing the limits of interpreting BM25 as a probability helps PMs work with search engineers on normalization, thresholding, and stakeholder-friendly explanations of ranking behavior.
• Evaluation and product tuning: BM25 can remain a strong baseline even when teams are excited about newer semantic methods. PMs can use it as a benchmark, identify failure modes where lexical retrieval outperforms embeddings, and prioritize tuning efforts that improve measurable search quality.

Related

• Doug Turnbull: Referenced as the source of recent discussion on BM25 score interpretation and practical normalization techniques.
• Hybrid search: Closely connected because BM25 is commonly used as the lexical component in systems that combine keyword-based and embedding-based retrieval.