llama.cpp Supports Vision Models! Plus Gemini 2.5 Pro & Local RAG with LangChain

AI Technology & Capabilities

• Vision Models & Multi-Modal Updates: Clement Delangue and Julien C highlighted that llama.cpp now supports vision models, featuring compatibility with models from Gemma, Pixtral, Qwen VL, and SmolVLM.

• Gemini’s Video Understanding: Demis Hassabis demonstrated Gemini 2.5 Pro’s exceptional video comprehension capabilities through AI Studio, encouraging users to test it with YouTube links.

• Token Caching Improvements: Jeff Dean announced that caching of tokens now happens implicitly when using the same input context repeatedly.

Product Management Tools & Strategies

• ProductBoard Analysis: Aakash G provided a comprehensive breakdown of how ProductBoard grew into a $1.7B platform, detailing its evolution from basic use cases to a comprehensive platform similar to Salesforce’s data model.

• Crisis Management Protocol: Nuri Janian shared a detailed crisis protocol for product managers handling critical product failures.

• Executive Communication: Nuri Janian outlined strategies for effective roadmap presentations to executives, emphasizing 70% discussion and 30% presentation ratio.

RAG & LLM Implementations

• Privacy-Focused RAG Solutions: LangChain introduced a fully local document Q&A system prioritizing privacy, and shared a winning RAG implementation for analyzing company annual reports.

• Structured Output Solutions: LangChain provided a guide for implementing structured output using Claude 3.7 through AWS Bedrock.

AI Research & Theory

• System Prompt Learning: Andrej Karpathy discussed the need for a new paradigm in LLM learning, suggesting “system prompt learning” as a potential approach distinct from pretraining and finetuning.

• Bayesian Thinking: Tiago Torres shared insights on applying Bayes’s Theorem to future predictions, emphasizing the importance of updating probability estimates with new information.

AI Product Recommendations

• Mender System: DeepLearning.AI highlighted research on Mender, a recommendation system using Llama 3 to infer customer preferences from text, outperforming traditional systems.

Memes & Humor

• Andrej Karpathy humorously noted: “Imagine you do 1 hour of intellectually difficult work just to learn that your grade is 0.32 lol”
• Logan K shared: “I love a good domain name : )”