GenAI PMDoug Turnbull
Search and retrieval expert mentioned for introducing pseudo-relevance feedback. He explains how early retrieval results can be used to refine queries.
Key Highlights
- Doug Turnbull is a recurring expert source on search, retrieval, and ranking systems relevant to AI products.
- He explains pseudo-relevance feedback as a way to use early retrieval results to improve subsequent search.
- His work helps teams choose among search engines, vector databases, and hybrid retrieval approaches.
- He also contributes practical guidance on BM25 calibration and LLM-based relevance evaluation.
- His writing is especially useful for AI PMs optimizing search quality, cost, and system reliability.
Doug Turnbull
Overview
Doug Turnbull is a search and retrieval expert frequently cited for practical explanations of information retrieval concepts, ranking methods, and evaluation techniques. In the newsletter, he appears as a recurring voice on topics such as BM25, pseudo-relevance feedback, retrieval engine selection, late interaction models, and LLM-based search relevance evaluation. For AI Product Managers building search, RAG, or knowledge retrieval products, his work is valuable because it translates core retrieval theory into concrete system design and decision-making guidance.He matters to AI PMs in particular because many modern AI products depend on retrieval quality more than model quality alone. Turnbull’s writing emphasizes the mechanics behind search performance: how to choose infrastructure, how to reason about lexical versus vector retrieval, how to evaluate ranking quality, and how to use early search results to improve subsequent retrieval. That makes his work especially relevant for PMs responsible for shipping reliable, explainable, and cost-effective retrieval systems.
Key Developments
- 2026-02-03 — Mentioned for Check twice, cut once with LLM search relevance eval, highlighting the importance of checking both directions in LLM pairwise evaluation for search relevance.
- 2026-03-07 — Mentioned for Can BM25 be a probability?, exploring BM25 through odds, probabilities, and a Bayesian framing with implications for hybrid search calibration.
- 2026-03-11 — Mentioned for The tests are the code now, arguing that AI-assisted coding increases the importance of tests as the primary quality safeguard.
- 2026-03-21 — Mentioned for How to actually choose a retrieval engine, comparing trade-offs across search engines and vector databases including Elasticsearch, OpenSearch, Solr, Vespa, Pinecone, Turbopuffer, and Weaviate.
- 2026-03-24 — Mentioned for Why tiny late interaction models win, discussing the rise of late interaction approaches and citing a LightOn demo with developer Antoine Chaffin using a 150M model.
- 2026-04-07 — Mentioned for Is grep all you need for RAG?, arguing that a RAG-like system can be built with simple tools given enough engineering effort, while warning about the practical difficulty.
- 2026-04-14 — Mentioned for What is pseudo-relevance feedback?, explaining how initial BM25 or ranked retrieval results can provide implicit feedback to refine queries and improve later retrieval stages.
Relevance to AI PMs
1. Better retrieval strategy decisions — Turnbull’s work helps PMs choose between lexical, vector, and hybrid approaches based on product constraints rather than hype. This is especially useful when selecting infrastructure or prioritizing relevance work for search and RAG features.2. Stronger evaluation and quality control — His writing on LLM pairwise relevance evaluation and tests-as-the-code gives PMs practical ways to improve measurement discipline. PMs can use these ideas to define better offline evals, reduce false confidence in ranking quality, and tighten acceptance criteria for AI-assisted systems.
3. Practical relevance optimization — Concepts like BM25 calibration, late interaction, and pseudo-relevance feedback give PMs tactical levers for improving retrieval without immediately resorting to larger or more expensive models. This is useful for teams optimizing latency, cost, and answer quality at the same time.
Related
- RAG — Turnbull is relevant to RAG because he discusses how retrieval can be implemented pragmatically, including minimal-tool approaches like grep and more sophisticated ranking systems.
- grep — Connected through his argument that grep can theoretically power RAG workflows, illustrating the trade-off between simplicity and engineering burden.
- LightOn and Antoine Chaffin — Referenced in his discussion of tiny late interaction models and their practical retrieval implications.
- Elasticsearch, OpenSearch, Solr, Vespa — Traditional search and retrieval engines he compares when advising teams on engine selection.
- Pinecone, Turbopuffer, Weaviate — Vector database options mentioned in the context of retrieval engine trade-offs.
- BM25 — A recurring topic in his work, especially around score interpretation, probabilistic framing, and hybrid search calibration.
- Hybrid search — Closely connected because his analysis often deals with combining lexical and semantic retrieval signals.
- LLM search relevance eval and LLM pairwise evaluation — Related through his emphasis on sound evaluation practices for ranking systems.
- Pseudo-relevance feedback — One of the clearest concepts associated with him in the newsletter, showing how early retrieval results can refine later search behavior.
- Tests-as-the-code — Connected through his broader perspective on quality and correctness in AI-assisted software development.
Newsletter Mentions (7)
“Doug Turnbull What is psuedo-relevance feedback? - Introduces pseudo-relevance feedback: after an initial BM25 or ranked retrieval, the returned results provide implicit information that can be used to refine queries or improve subsequent retrieval.”
#7 📝 Doug Turnbull What is psuedo-relevance feedback? - Introduces pseudo-relevance feedback: after an initial BM25 or ranked retrieval, the returned results provide implicit information that can be used to refine queries or improve subsequent retrieval. The post outlines how to leverage those initial results as a source of feedback to boost relevance.
“#11 📝 Doug Turnbull Is grep all you need for RAG? - Doug argues that with enough engineering effort you can build a RAG-style search system using only grep, but cautions that this approach is difficult and not for the faint of heart.”
#11 📝 Doug Turnbull Is grep all you need for RAG? - Doug argues that with enough engineering effort you can build a RAG-style search system using only grep, but cautions that this approach is difficult and not for the faint of heart.
“Doug Turnbull Why tiny late interaction models win - Discusses the recent prominence of late interaction models, highlighting a LightOn demonstration (with developer Antoine Chaffin) using a 150M model and the implications for retrieval and interaction approaches.”
#12 📝 Doug Turnbull Why tiny late interaction models win - Discusses the recent prominence of late interaction models, highlighting a LightOn demonstration (with developer Antoine Chaffin) using a 150M model and the implications for retrieval and interaction approaches.
“Doug Turnbull How to actually choose a retrieval engine - Explains how teams should choose a retrieval engine by comparing vector databases and search engines, and considering trade-offs between options like Elasticsearch, OpenSearch, Solr, Vespa, Pinecone, Turbopuffer, and Weaviate.”
#5 📝 Doug Turnbull How to actually choose a retrieval engine - Explains how teams should choose a retrieval engine by comparing vector databases and search engines, and considering trade-offs between options like Elasticsearch, OpenSearch, Solr, Vespa, Pinecone, Turbopuffer, and Weaviate. Emphasizes practical selection criteria beyond hype.
“#15 📝 Doug Turnbull The tests are the code now - Argues that with AI-assisted coding, tests become the most important artifact for maintaining code quality.”
Doug Turnbull is cited for a piece arguing that AI-assisted coding elevates the importance of tests. The newsletter uses him to support a broader software quality point.
“#20 📝 Doug Turnbull Can BM25 be a probability? - Explores the relationship between BM25 scores framed as odds versus probabilities and introduces a Bayesian view of BM25.”
GenAI PM Daily March 07, 2026 GenAI PM Daily 🎧 Listen to this brief 3 min listen Today's top 25 insights for PM Builders, ranked by relevance from LinkedIn, YouTube, X, and Blogs. #20 📝 Doug Turnbull Can BM25 be a probability? - Explores the relationship between BM25 scores framed as odds versus probabilities and introduces a Bayesian view of BM25. Discusses implications for calibrating hybrid search systems when combining lexical and probabilistic signals.
“Check twice, cut once with LLM search relevance eval - Highlights the importance of checking both directions in LLM pairwise evaluation of search relevance.”
GenAI PM Daily February 03, 2026 GenAI PM Daily Today's top 10 insights for PM Builders, ranked by relevance from Blogs, X, YouTube, and LinkedIn. OpenAI Launches Codex App 📝 OpenAI News Introducing the Codex app - OpenAI has launched the Codex app, enhancing user interaction with AI. Read more → 𝕏 claire vo 🖤 @clairevo Claire overhauled Maplewood’s architecture by migrating to Inngest workflows and persisting stories/actions in NeonDB, added infinite scroll for event feeds, and squashed an auto-scroll bug. Read more → 📝 Doug Turnbull Check twice, cut once with LLM search relevance eval - Highlights the importance of checking both directions in LLM pairwise evaluation of search relevance.
Related
RAG is a retrieval-based pattern that injects external context into prompts to improve model responses. The newsletter presents it as often outperforming fine-tuning for practical product work.
A lexical retrieval ranking function used here to select relevant tool definitions. In PM tooling, it helps improve retrieval accuracy and reduce context-window bloat.
Elasticsearch is referenced in the context of hybrid search and kNN query behavior in practice.
Stay updated on Doug Turnbull
Get curated AI PM insights delivered daily — covering this and 1,000+ other sources.
Subscribe Free