Doug Turnbull

Overview

Doug Turnbull is a search and retrieval expert whose writing is frequently cited for practical explanations of how modern information retrieval systems work. In the newsletter, he appears most often in discussions about search relevance, retrieval engine trade-offs, BM25, hybrid search, LLM-based relevance evaluation, and pseudo-relevance feedback. His work is especially useful because it connects classic search concepts with current AI product concerns such as RAG quality, retrieval calibration, and evaluation methodology.

For AI Product Managers, Turnbull matters as a translator between search theory and implementation reality. Rather than focusing on hype, his posts emphasize how retrieval systems actually behave in production: how to choose the right engine, how to think about lexical versus vector retrieval, how to evaluate ranking quality, and how to use early search results to improve downstream relevance. That makes his perspective highly relevant for teams building search, recommendation, knowledge retrieval, and RAG-powered products.

Key Developments

• 2026-02-03 — Mentioned for Check twice, cut once with LLM search relevance eval, highlighting the importance of checking both directions in LLM pairwise evaluation when assessing search relevance.
• 2026-03-07 — Mentioned for Can BM25 be a probability?, exploring BM25 through a Bayesian lens and discussing implications for score interpretation and hybrid search calibration.
• 2026-03-11 — Mentioned for The tests are the code now, arguing that in AI-assisted coding workflows, tests become the key artifact for preserving software quality.
• 2026-03-21 — Mentioned for How to actually choose a retrieval engine, comparing search engines and vector databases including Elasticsearch, OpenSearch, Solr, Vespa, Pinecone, Turbopuffer, and Weaviate.
• 2026-03-24 — Mentioned for Why tiny late interaction models win, discussing the rise of late interaction retrieval approaches and highlighting a LightOn demonstration involving Antoine Chaffin and a 150M model.
• 2026-04-07 — Mentioned for Is grep all you need for RAG?, arguing that a RAG-like search system can be built with enough engineering effort using simple tooling, while stressing the practical difficulty of doing so.
• 2026-04-14 — Mentioned for What is pseudo-relevance feedback?, explaining how initial ranked retrieval results can be used as implicit feedback to refine a query and improve subsequent retrieval.

Relevance to AI PMs

1. Improving RAG retrieval quality Turnbull’s work helps PMs understand when better relevance comes from query reformulation, BM25 tuning, hybrid retrieval, or pseudo-relevance feedback rather than simply swapping in a new model. This is useful when diagnosing weak answer grounding, poor recall, or repetitive failure cases in RAG systems.

2. Making better infrastructure choices
His comparisons of retrieval engines provide a practical framework for choosing between classic search stacks and vector-first systems. PMs can use these criteria to align technical decisions with product requirements such as latency, filtering, ranking flexibility, operational complexity, and cost.

3. Designing stronger evaluation loops
His emphasis on LLM relevance evaluation and test quality is directly applicable to PMs building search or AI copilots. Teams can use these ideas to create more reliable offline evals, pairwise ranking checks, and regression tests before shipping retrieval changes.

Related

• RAG — Turnbull’s retrieval insights are highly relevant to retrieval-augmented generation, especially around grounding quality and system design trade-offs.
• grep — Used in his argument that simple tools can approximate RAG pipelines with enough engineering effort, illustrating the trade-off between minimalism and maintainability.
• LightOn and Antoine Chaffin — Connected through discussion of tiny late interaction models and efficient retrieval approaches.
• Elasticsearch, OpenSearch, Solr, Vespa — Traditional or hybrid-capable search engines featured in his retrieval engine selection guidance.
• Pinecone, Turbopuffer, Weaviate — Vector database options discussed in contrast to search engines when choosing retrieval infrastructure.
• BM25 — A recurring theme in his work, especially in relation to score interpretation, relevance ranking, and hybrid search.
• Hybrid search — Central to his framing of combining lexical and probabilistic or vector signals effectively.
• LLM search relevance eval and LLM pairwise evaluation — Related to his guidance on evaluating ranking quality and avoiding one-sided comparison errors.
• Pseudo-relevance feedback — One of the clearest concepts associated with Turnbull in these mentions, showing how initial retrieval outputs can improve later retrieval steps.
• Tests-as-the-code — Connects his search thinking to broader AI engineering practice, especially quality assurance in AI-assisted software development.