GenAI PMreinforcement learning
A training approach where a model learns behavior from feedback or trajectories. The newsletter references it as part of an enterprise architecture for learning good and bad outcomes from business actions.
Key Highlights
- Reinforcement learning helps AI systems improve behavior based on outcome feedback rather than prompts alone.
- Cursor used RL to train Composer to self-summarize, reportedly cutting compaction errors by 50%.
- Kimi Moonshot, Cursor, and Chroma were highlighted as training vertical agentic models with RL.
- For AI PMs, RL is most valuable when tied to measurable product outcomes like task success and error reduction.
Reinforcement learning
Overview
Reinforcement learning (RL) is a training approach where a model learns to make better decisions by receiving feedback on the outcomes of its actions, rather than only imitating examples or following fixed prompts. In AI products, RL is especially useful when success depends on multi-step behavior, adaptation, and optimizing for real-world results such as task completion, correctness, or reduced error rates.For AI Product Managers, RL matters because it offers a path to improve agent reliability and product performance beyond prompt engineering alone. The newsletter mentions show RL being used in two important ways: first, to train Composer to self-summarize instead of relying on handcrafted prompts, reducing compaction errors by 50%; and second, as a method used by companies like Kimi Moonshot, Cursor, and Chroma to train vertical agentic models on top of strong base models with production harnesses and outcome-based rewards. This makes RL highly relevant for products that need durable, measurable improvements in complex workflows.
Key Developments
- 2026-03-18: Cursor trained Composer to self-summarize via reinforcement learning instead of relying on a prompt. According to the newsletter mention, this cut compaction errors by 50% and helped Composer handle coding tasks requiring hundreds of actions.
- 2026-03-29: Philipp Schmid highlighted that Kimi Moonshot, Cursor, and Chroma train vertical agentic models via RL by combining a strong base model, a production harness, and outcome-based rewards. This framed RL as a practical recipe for improving domain-specific AI agents.
Relevance to AI PMs
- Improve product behavior beyond prompt tuning: RL can replace brittle prompt-based control with learned behaviors, which may lead to more consistent performance in long-running or multi-step tasks.
- Tie model improvement to product metrics: Outcome-based rewards let PMs align training with business-relevant goals such as task success, reduced error rates, higher completion rates, or better user satisfaction.
- Build defensible vertical agents: The combination of a capable base model, production environment data, and RL can help teams create specialized agents that perform better in real workflows than generic models alone.
Related
- Kimi Moonshot: Mentioned as one of the companies training vertical agentic models with RL, suggesting RL is part of its applied model improvement strategy.
- Cursor: Featured prominently in both mentions: as a company using RL in vertical agents and as the team that trained Composer to self-summarize through RL.
- Chroma: Included alongside Kimi Moonshot and Cursor as an example of a company applying RL to vertical agentic systems.
- Composer: A concrete product example where RL was used to train self-summarization behavior, reducing errors and enabling longer coding workflows.
Newsletter Mentions (5)
“He also described an Intuit architecture where frontier models propose actions while a reinforcement-learning model learns from millions of business trajectories what produces good or harmful outcomes.”
#15 in Udi Menkes recapped how Alex Karp linked Palantir Technologies’ growth this week to open-weight models and in-organization fine-tuning, with customers retaining the weights and resulting “alpha” from task-specific models that can outperform frontier models. He also described an Intuit architecture where frontier models propose actions while a reinforcement-learning model learns from millions of business trajectories what produces good or harmful outcomes.
“Udi Menkes 🚢 shared AI Engineer’s video of his talk on Intuit’s financial-advice system, which derives millions of business state–action–outcome trajectories, uses reinforcement learning to select better actions, and trains an LLM to generate advice. Intuit reported that a cheaper midsize model grounded in this data outperformed leading models.”
#1 𝕏 Udi Menkes 🚢 shared AI Engineer’s video of his talk on Intuit’s financial-advice system, which derives millions of business state–action–outcome trajectories, uses reinforcement learning to select better actions, and trains an LLM to generate advice. Intuit reported that a cheaper midsize model grounded in this data outperformed leading models.
“Hugging Face unveiled Training Agents 3, demonstrating how to train a local, open-weight agent from scratch using reinforcement learning.”
#13 𝕏 Hugging Face unveiled Training Agents 3, demonstrating how to train a local, open-weight agent from scratch using reinforcement learning.
“#2 𝕏 Philipp Schmid shows how @Kimi_Moonshot, @cursor_ai, and @trychroma all train vertical agentic models via RL using a strong base model, production harness, and outcome-based rewards.”
Today's top 10 insights for PM Builders from X and Blogs. #2 𝕏 Philipp Schmid shows how @Kimi_Moonshot, @cursor_ai, and @trychroma all train vertical agentic models via RL using a strong base model, production harness, and outcome-based rewards. K2.
“Cursor trained Composer to self-summarize via reinforcement learning instead of relying on a prompt, cutting compaction errors by 50% and enabling it to tackle coding tasks requiring hundreds of actions.”
#4 𝕏 Cursor trained Composer to self-summarize via reinforcement learning instead of relying on a prompt, cutting compaction errors by 50% and enabling it to tackle coding tasks requiring hundreds of actions.
Related
An AI coding environment used for software development and workspace-based agent workflows.
A large language model used as the reasoning core inside agents and tool-calling systems. PMs often evaluate LLMs based on orchestration, context loading, and task execution behavior.
Cursor’s agentic coding assistant/model referenced as part of the Cursor Start plan. For PMs, it is part of the developer productivity and autonomous coding stack.
Stay updated on reinforcement learning
Get curated AI PM insights delivered daily — covering this and 1,000+ other sources.
Subscribe Free