BM25

Overview

BM25 is a classic lexical retrieval ranking function used to score how relevant a document is to a query based on term frequency, document frequency, and document length. In AI product systems, it is often used to retrieve the most relevant items from a corpus before an LLM sees them—such as tool definitions, knowledge snippets, or internal documentation.

For AI Product Managers, BM25 matters because it offers a practical, fast, and interpretable way to improve retrieval quality without requiring embeddings alone. In PM tooling, it can reduce context-window bloat by loading only the most relevant information, which improves tool selection, lowers token costs, and can increase reliability in agentic workflows. It also plays an important role in hybrid search systems, where lexical matching is combined with semantic retrieval or probabilistic scoring.

Key Developments

• 2026-03-07 — Doug Turnbull’s note Can BM25 be a probability? explored whether BM25 scores can be interpreted as odds or probabilities, introducing a Bayesian framing relevant to hybrid search calibration.
• 2026-03-10 — Doug Turnbull’s Ugly hack to force BM25 to 0-1 described a pragmatic normalization technique for converting BM25 lexical scores into a 0–1 range, making them easier to compare with other retrieval signals.
• 2026-04-20 — In Claude Design’s export-to-Canva feature, the Tool Search Tool used BM25 to load only relevant tool definitions, reducing context-window usage and improving tool-selection accuracy.

Relevance to AI PMs

• Improve retrieval accuracy in agent workflows: BM25 helps rank exact-match and keyword-relevant documents, which is especially useful when an assistant must choose among many tools, APIs, or internal actions.
• Reduce token usage and context bloat: By retrieving only the most relevant definitions or documents before passing them to an LLM, BM25 can lower inference costs and improve model focus.
• Support hybrid search design: PMs working on RAG, copilots, or tool routers can use BM25 alongside embeddings to balance precision, recall, explainability, and score calibration.

Related

• Doug Turnbull — Frequently referenced in connection with BM25 scoring, normalization, and probabilistic interpretations relevant to search system design.
• Hybrid search — BM25 is commonly paired with vector search in hybrid retrieval systems to combine lexical precision with semantic recall.
• Tool Search Tool — An example application where BM25 powers retrieval of relevant tool definitions before they are loaded into model context.
• Claude Design — Product example where BM25-backed tool retrieval improved export workflow efficiency and tool-selection accuracy.