GenAI PMDoug Turnbull
Search and retrieval expert mentioned for introducing pseudo-relevance feedback. He explains how early retrieval results can be used to refine queries.
Key Highlights
- Doug Turnbull is a recurring expert voice on search, retrieval, and relevance tuning for modern AI products.
- He explains pseudo-relevance feedback as a way to use initial search results to refine later retrieval.
- His work helps AI PMs choose between search engines and vector databases using practical trade-offs.
- He connects classical IR concepts like BM25 to modern hybrid search and RAG system design.
- He also contributes useful guidance on LLM-based search relevance evaluation and AI-assisted coding quality.
Overview
Doug Turnbull is a search and retrieval expert whose writing frequently focuses on practical information retrieval decisions, relevance tuning, and evaluation. In the newsletter, he appears as a source of grounded thinking on topics such as BM25, pseudo-relevance feedback, retrieval engine selection, search evaluation, and the engineering trade-offs behind RAG systems. For AI Product Managers, his work is useful because it connects classic search principles with modern LLM and retrieval workflows.His relevance comes from helping teams move beyond hype and make better product and infrastructure decisions. Across the cited mentions, Turnbull explains how to improve query quality using early retrieval results, how to think about lexical ranking signals like BM25, how to evaluate search systems with LLMs more carefully, and how to choose among engines such as Elasticsearch, OpenSearch, Solr, Vespa, Pinecone, Turbopuffer, and Weaviate. He is especially valuable for PMs building search, RAG, agent, or knowledge retrieval products where ranking quality and system choice directly affect user outcomes.
Key Developments
- 2026-02-03 — Mentioned for Check twice, cut once with LLM search relevance eval, emphasizing that LLM pairwise evaluation for search relevance should be checked in both directions to avoid misleading judgments.
- 2026-03-07 — Mentioned for Can BM25 be a probability?, exploring BM25 through odds, probabilities, and a Bayesian framing, with implications for score calibration in hybrid search.
- 2026-03-11 — Mentioned for The tests are the code now, arguing that in AI-assisted coding workflows, tests become a central quality mechanism.
- 2026-03-21 — Mentioned for How to actually choose a retrieval engine, comparing search engines and vector databases including Elasticsearch, OpenSearch, Solr, Vespa, Pinecone, Turbopuffer, and Weaviate, and stressing practical trade-offs.
- 2026-03-24 — Mentioned for Why tiny late interaction models win, discussing the rise of late interaction approaches and referencing a LightOn demo with developer Antoine Chaffin using a 150M model.
- 2026-04-07 — Mentioned for Is grep all you need for RAG?, arguing that a RAG-like system can be built with simple tools given enough engineering effort, while cautioning that this is hard and rarely the easiest path.
- 2026-04-14 — Mentioned for What is psuedo-relevance feedback?, explaining pseudo-relevance feedback as a technique where initial BM25 or ranked retrieval results provide implicit signals to refine queries and improve subsequent retrieval.
Relevance to AI PMs
1. Improving RAG and search quality — Turnbull's work helps PMs understand that better retrieval is often achieved through ranking, query reformulation, and evaluation discipline rather than only changing models. This is especially useful when diagnosing why a RAG product returns incomplete or off-topic answers.2. Making smarter infrastructure choices — His comparisons of retrieval engines and vector databases provide a practical framework for choosing systems based on workload, relevance needs, team capability, and operational trade-offs instead of vendor hype.
3. Designing evaluation and iteration loops — His writing on BM25 calibration, LLM pairwise relevance evaluation, and pseudo-relevance feedback gives PMs concrete ideas for building measurement and improvement loops into search and retrieval products.
Related
- RAG — Turnbull's writing connects retrieval fundamentals to RAG system design, especially around whether simple retrieval approaches can be enough.
- grep — Used as a provocative example in his discussion of building retrieval pipelines with minimal tooling.
- BM25 — A recurring theme in his work, especially for explaining lexical relevance, calibration, and hybrid search trade-offs.
- Hybrid search — His BM25 and retrieval engine discussions are directly relevant to combining lexical and vector signals.
- Pseudo-relevance feedback — One of the clearest concepts associated with him in these mentions, showing how first-pass results can improve later retrieval.
- LLM search relevance eval / LLM pairwise evaluation — He is cited on best practices for evaluating retrieval quality with LLM judges.
- Elasticsearch, OpenSearch, Solr, Vespa — Core search engine options he discusses when advising teams on retrieval stack selection.
- Pinecone, Turbopuffer, Weaviate — Vector database and retrieval platform options included in his comparison framework.
- LightOn and Antoine Chaffin — Connected through his discussion of tiny late interaction models and emerging retrieval model design patterns.
- tests-as-the-code — Related through his argument that tests become the critical artifact in AI-assisted software development.
Newsletter Mentions (7)
“Doug Turnbull What is psuedo-relevance feedback? - Introduces pseudo-relevance feedback: after an initial BM25 or ranked retrieval, the returned results provide implicit information that can be used to refine queries or improve subsequent retrieval.”
#7 📝 Doug Turnbull What is psuedo-relevance feedback? - Introduces pseudo-relevance feedback: after an initial BM25 or ranked retrieval, the returned results provide implicit information that can be used to refine queries or improve subsequent retrieval. The post outlines how to leverage those initial results as a source of feedback to boost relevance.
“#11 📝 Doug Turnbull Is grep all you need for RAG? - Doug argues that with enough engineering effort you can build a RAG-style search system using only grep, but cautions that this approach is difficult and not for the faint of heart.”
#11 📝 Doug Turnbull Is grep all you need for RAG? - Doug argues that with enough engineering effort you can build a RAG-style search system using only grep, but cautions that this approach is difficult and not for the faint of heart.
“Doug Turnbull Why tiny late interaction models win - Discusses the recent prominence of late interaction models, highlighting a LightOn demonstration (with developer Antoine Chaffin) using a 150M model and the implications for retrieval and interaction approaches.”
#12 📝 Doug Turnbull Why tiny late interaction models win - Discusses the recent prominence of late interaction models, highlighting a LightOn demonstration (with developer Antoine Chaffin) using a 150M model and the implications for retrieval and interaction approaches.
“Doug Turnbull How to actually choose a retrieval engine - Explains how teams should choose a retrieval engine by comparing vector databases and search engines, and considering trade-offs between options like Elasticsearch, OpenSearch, Solr, Vespa, Pinecone, Turbopuffer, and Weaviate.”
#5 📝 Doug Turnbull How to actually choose a retrieval engine - Explains how teams should choose a retrieval engine by comparing vector databases and search engines, and considering trade-offs between options like Elasticsearch, OpenSearch, Solr, Vespa, Pinecone, Turbopuffer, and Weaviate. Emphasizes practical selection criteria beyond hype.
“#15 📝 Doug Turnbull The tests are the code now - Argues that with AI-assisted coding, tests become the most important artifact for maintaining code quality.”
Doug Turnbull is cited for a piece arguing that AI-assisted coding elevates the importance of tests. The newsletter uses him to support a broader software quality point.
“#20 📝 Doug Turnbull Can BM25 be a probability? - Explores the relationship between BM25 scores framed as odds versus probabilities and introduces a Bayesian view of BM25.”
GenAI PM Daily March 07, 2026 GenAI PM Daily 🎧 Listen to this brief 3 min listen Today's top 25 insights for PM Builders, ranked by relevance from LinkedIn, YouTube, X, and Blogs. #20 📝 Doug Turnbull Can BM25 be a probability? - Explores the relationship between BM25 scores framed as odds versus probabilities and introduces a Bayesian view of BM25. Discusses implications for calibrating hybrid search systems when combining lexical and probabilistic signals.
“Check twice, cut once with LLM search relevance eval - Highlights the importance of checking both directions in LLM pairwise evaluation of search relevance.”
GenAI PM Daily February 03, 2026 GenAI PM Daily Today's top 10 insights for PM Builders, ranked by relevance from Blogs, X, YouTube, and LinkedIn. OpenAI Launches Codex App 📝 OpenAI News Introducing the Codex app - OpenAI has launched the Codex app, enhancing user interaction with AI. Read more → 𝕏 claire vo 🖤 @clairevo Claire overhauled Maplewood’s architecture by migrating to Inngest workflows and persisting stories/actions in NeonDB, added infinite scroll for event feeds, and squashed an auto-scroll bug. Read more → 📝 Doug Turnbull Check twice, cut once with LLM search relevance eval - Highlights the importance of checking both directions in LLM pairwise evaluation of search relevance.
Related
RAG is a retrieval-based pattern that injects external context into prompts to improve model responses. The newsletter presents it as often outperforming fine-tuning for practical product work.
A lexical retrieval ranking function used here to select relevant tool definitions. In PM tooling, it helps improve retrieval accuracy and reduce context-window bloat.
Elasticsearch is referenced in the context of hybrid search and kNN query behavior in practice.
Stay updated on Doug Turnbull
Get curated AI PM insights delivered daily — covering this and 1,000+ other sources.
Subscribe Free