Doug Turnbull

Overview

Doug Turnbull is a search relevance expert whose work shows up repeatedly in discussions about retrieval quality, ranking systems, and practical evaluation methods for AI-powered search. For AI Product Managers, he is especially relevant because his writing bridges classic information retrieval ideas—like BM25, hybrid search, and engine selection—with modern LLM-era concerns such as pairwise evaluation and retrieval design for RAG systems.

Across the newsletter mentions, Turnbull appears as a pragmatic voice focused on how search systems actually perform in production. His perspectives matter to AI PMs because they help teams move beyond hype: choosing the right retrieval stack, understanding scoring trade-offs, using late interaction models thoughtfully, and evaluating search relevance carefully by checking comparisons in both directions.

Key Developments

• 2026-02-03 — Mentioned for "Check twice, cut once with LLM search relevance eval", emphasizing that LLM pairwise evaluation should test both directions rather than assuming a single comparison is reliable.
• 2026-03-07 — Featured for "Can BM25 be a probability?", exploring BM25 through odds, probabilities, and a Bayesian framing with implications for calibrated hybrid search.
• 2026-03-11 — Cited in "The tests are the code now", arguing that in AI-assisted coding workflows, tests increasingly become the core artifact for preserving software quality.
• 2026-03-21 — Highlighted for "How to actually choose a retrieval engine", comparing trade-offs among search engines and vector databases such as Elasticsearch, OpenSearch, Solr, Vespa, Pinecone, Turbopuffer, and Weaviate.
• 2026-03-24 — Referenced in "Why tiny late interaction models win", discussing the growing importance of late interaction retrieval and a LightOn demonstration involving Antoine Chaffin and a compact 150M model.
• 2026-04-07 — Mentioned in "Is grep all you need for RAG?", arguing that a RAG-like search system can be built with enough engineering effort using simpler primitives, while cautioning that this path is difficult and rarely the easy answer.

Relevance to AI PMs

1. Improves retrieval and RAG decision-making Turnbull’s work helps AI PMs evaluate when to use traditional search engines, vector databases, or hybrid architectures. This is useful when defining product requirements for retrieval quality, latency, cost, and maintainability.

2. Provides better evaluation discipline for LLM features
His emphasis on pairwise evaluation and checking both directions is highly practical for PMs designing offline evals for search, ranking, recommendation, or answer quality. It reduces the risk of overtrusting noisy LLM judgments.

3. Connects classical IR concepts to modern AI systems
By discussing BM25, late interaction, hybrid search, and engine trade-offs, Turnbull gives PMs a framework for talking with engineers about why retrieval quality changes and how to prioritize experiments that materially improve user outcomes.

Related

• RAG — Turnbull’s views on retrieval design and even minimalist approaches like grep-based systems connect directly to how RAG products are built.
• grep — Referenced as a provocative example of how far simple tooling can go with enough engineering effort in retrieval pipelines.
• LightOn — Appears in the discussion of tiny late interaction models, illustrating efficient retrieval approaches.
• Antoine Chaffin — Connected through the LightOn late interaction demonstration referenced alongside Turnbull’s commentary.
• Elasticsearch, OpenSearch, Solr, Vespa — Core search engine options discussed in retrieval engine selection.
• Pinecone, Turbopuffer, Weaviate — Vector database and retrieval infrastructure options considered in build-vs-buy and architecture decisions.
• tests-as-the-code — Linked to Turnbull’s argument that tests become the key artifact in AI-assisted software development.
• BM25 — A central ranking concept in his writing, especially for understanding lexical retrieval and hybrid scoring.
• hybrid-search — Closely connected to his work on combining lexical and semantic retrieval signals.
• llm-search-relevance-eval and llm-pairwise-evaluation — Directly tied to his emphasis on careful, bidirectional evaluation of LLM-based search judgments.