LLMs: 39% Performance Drop in Multi-Turn Conv, Microsoft's New Reasoning Models

AI Development & Research Updates

• LLM Performance in Multi-Turn Conversations: Philipp Schmid @_philschmid shared research showing a 39% performance drop in multi-turn conversations, with a 112% increase in unreliability. Key findings include LLMs making premature assumptions and struggling to recover from early mistakes.

• Microsoft’s New Reasoning Models: DeepLearning.AI reported on Microsoft’s release of three open-weight reasoning models (Phi-4-reasoning series) with detailed training methods, outperforming peers on math tasks under MIT license.

• Improved LLM Reasoning: A new study shared by DeepLearning.AI showed significant improvement in LLM reasoning by fine-tuning on just 1,000 examples, achieving strong performance on AIME and MATH 500 benchmarks.

Product Launches & Updates

• OpenAI’s Codex Launch: Sam Altman @sama announced Codex, a cloud-based software engineering agent that can handle multiple tasks in parallel, rolling out to ChatGPT Pro, Enterprise, and Team users.

• Google’s Media Generation Updates: Logan Kilpatrick @OfficialLoganK introduced new generative media experience in Google AI Studio, combining Veo 2, Gemini 2.0, and Imagen 3 for image generation and editing.

• Qwen2.5 Model Release: Alibaba Qwen released quantized versions of Qwen2.5-Omni-7B models on Hugging Face and ModelScope.

Product Management Insights

• Remote Work Trends: Lenny Rachitsky @lennysan shared data showing remote work has settled at about 20% of open roles, down from a peak of 33% in late 2022.

• PM Best Practices: Nuri Janian @nurijanian provided insights on proper MVP development, focusing on proving value rather than just cutting features.

Memes & Humor

• Sam Altman @sama shared amusing contrasting reactions to Codex pricing, with users split between considering $20/month too expensive or too cheap for a software engineering tool.