“#4 𝕏 Boris Cherny said Claude has handled an unspecified task better than he could since around Opus 4.8, and that 5.5 models are now very good at it.”

#4 𝕏 Boris Cherny said Claude has handled an unspecified task better than he could since around Opus 4.8, and that 5.5 models are now very good at it. His workflow is to provide a goal, a way to measure results, and guidance on effort and duration—potentially a few weeks.

“Santiago recapped a Sapient Intelligence post about the PRAXIST Beta launch, reporting 49 gold medals across 75 tasks versus 35 for Claude Code using Opus 4.8, at ~$3K in token costs versus ~$38K—roughly one-twelfth the cost.”

#15 𝕏 Santiago recapped a Sapient Intelligence post about the PRAXIST Beta launch, reporting 49 gold medals across 75 tasks versus 35 for Claude Code using Opus 4.8, at ~$3K in token costs versus ~$38K—roughly one-twelfth the cost. Users define a measurable goal and evaluator, while multiple agents explore, test and refine approaches before providing a solution with supporting evidence.

“Shreya uses the free Error Discovery Skill in Claude Code with Opus 4.8 to turn human feedback on AI-generated writing into annotated failure modes, an evaluation rubric, and reusable LLM-judge criteria.”

#1 ▶️ How to Build Better AI Evals with Claude Code in 5 Steps | Shreya & Hamel Peter Yang Shreya uses the free Error Discovery Skill in Claude Code with Opus 4.8 to turn human feedback on AI-generated writing into annotated failure modes, an evaluation rubric, and reusable LLM-judge criteria. The Error Discovery Skill performs five steps: identifies the dataset’s semantic type, designs visual encoding, builds an HTML/Python review app, clusters data and selects diverse samples, then uses an interactive feedback loop to propose new samples and rubric criteria. In the live run, Claude took about 15 minutes to build a three-tab review interface with article-by-article, map/clustering, and progress views; after feedback was supplied, it generated 361 suggested annotations, including 249 instances of the “less than four words staccato” rule. Hamel’s benchmark found automated-eval tools such as Braintrust Loop, Arize Alex, and LangSmith recovered many obvious failures but missed product-judgment failures such as unhandled sales objections; their best-case precision was stated as 80% to 90%, meaning 10% to 20% of flagged errors were not actual errors.

“He also expects an Opus 4.8–grade model to run locally on devices within a year.”

#15 𝕏 Aravind Srinivas predicts a >50% chance we’ll have a Fable 5–quality model at 3–4× lower cost in under six months. He also expects an Opus 4.8–grade model to run locally on devices within a year.

“It’s available in beta today for Claude Enterprise and Team customers (runs on Opus 4.8), admins can control per-channel permissions, token spend limits and activity logs, migration from the prior Claude-in-Slack app is opt-in within 30 days, and Anthropic reports 65% of its product team’s code is created by their internal Claude Tag.”

GenAI PM Daily June 24, 2026 GenAI PM Daily 🎧 Listen to this brief 3 min listen Today's top 25 insights for PM Builders, ranked by relevance from Blogs, X, YouTube, and LinkedIn. Anthropic launches Claude Tag Slack AI assistant #1 📝 Anthropic News Introducing Claude Tag - Claude Tag is a Slack-integrated Claude that joins workspaces as a team member, can be granted channel- and tool-specific access, remembers channel context, breaks requests into staged tasks, schedules and pursues work asynchronously, and can proactively surface updates in an "ambient" mode. It’s available in beta today for Claude Enterprise and Team customers (runs on Opus 4.8), admins can control per-channel permissions, token spend limits and activity logs, migration from the prior Claude-in-Slack app is opt-in within 30 days, and Anthropic reports 65% of its product team’s code is created by their internal Claude Tag. Also covered by: @Claude , @Claude , @Boris Cherny , @Thariq #2 𝕏 NVIDIA AI launched DFlash, an open-source lightweight block diffusion model for speculative decoding that delivers up to 15× higher inference throughput on NVIDIA Blackwell without sacrificing responsiveness. #3 𝕏 Mistral AI launched Mistral OCR 4, a structured OCR system offering bounding boxes, block classification, and inline confidence scores across 170 languages. #4 📝 Claude Code Blog Agent identity in Claude Tag: a new access model for autonomous, team-wide AI - Introduces agent identity in Claude Tag as a new access model that enables teams to run autonomous agents with team-wide access controls. The post describes a model for managing agent identities and permissions to support safer, scalable agent deployments. Also covered by: @Claude , @Claude , @Boris Cherny , @Thariq #5 𝕏 Santiago shows how Claude Code plus Apify actors and MCP connectors can fetch and interact with any web content (even behind paywalls) to automate tasks. You can, in seconds, link Notion, Google Calendar, etc., to auto-summarize YouTube videos or import school events. #6 𝕏 Philipp Schmid published a developer guide for the Gemini Interactions API, covering streaming responses, conversation chaining via `previous_interaction_id`, tool use, and managed agents. #7 𝕏 Harrison Chase says that shipping an AI agent is just the start. A reliable agent requires a repeatable 5-step cycle—Build, Test, Deploy, Monitor and Improve—to iteratively refine prompts, tools and workflows based on real-world usage. #8 𝕏 Harrison Chase unveils Self-Harness: a DeepAgents-based framework where agents mine their own failure modes, propose harness tweaks, and regression-test those changes to auto-improve over time. #9 📝 Surge AI Blog HANDBOOK.md — Can Agents Follow 100-Page Company Policies? - Introduces HANDBOOK.md, a benchmark for long-context enterprise agents that tests capability to follow expert-written company handbooks up to 124 pages. Results show no frontier model exceeds 25% and some agents acted incorrectly (firing employees) while reporting compliance. #10 ▶️ GLM 5.2: Set Up Local AI with Cursor/Codex etc Greg Isenberg Sets up GLM 5.2 from Z AI in Cursor and Codex via OpenRouter and sequences it with Opus 4.8 and Composer 2.5 to optimize performance and cost. GLM 5.2 provides a 1 million-token context window and scores 81 on Terminal Bench 2.1, about four points behind Opus 4.8. A 50 000-input + 85 000-output token task via OpenRouter on GLM 5.2 costs $0.44, versus $2.38 on Opus 4.8. Set up GLM 5.2 by pasting a Z AI API key into Cursor's OpenAI field and overriding the endpoint, or by creating a Codex profile with an OpenRouter key and switching to GLM 5.2 via CLI. #11 𝕏 clem 🤗 – Co-founder & CEO @HuggingFace details how LeRobot integrates with Hugging Face Storage Buckets to deliver infinite, append-only storage for colossal robotics and video AI datasets—public or private. #12 in Marc Baselga highlights Kyler Ross’s iTerm “operator desk” setup—running Cloaked and PMAI agents in parallel across tabs for PM, exec, and management workstreams to keep context visible and resumable. #13 📝 HumanLayer Blog Announcing general availability for HumanLayer and HumanLayer Cloud→ - HumanLayer and HumanLayer Cloud are now generally available as an AI coding IDE and collaboration platform that the company says lets engineers ship 2–3x faster across the SDLC while maintaining code quality, and it supports “bring your own” AI subscriptions (Claude, Codex, etc.) with no separate per‑token billing. The platform groups tasks, agent sessions, artifacts and worktrees in collaborative workspaces, runs agents via a local daemon or cloud daemons with a unified web/desktop/mobile UI, and enforces a six‑phase Q‑R‑D‑S‑P‑I workflow (Questions, Research, Design, Structure, Plan, Implement) with comment‑driven design reviews. #14 𝕏 bolt.new reveals that top-earning real estate entrepreneurs on Bolt run their businesses with self-built listing tools, lead trackers, and client portals—and offers a video tutorial plus templates to help you build your own. #15 📝 Armin Ronacher The Coming Loop - I haven’t had much success using harness-level loops for code I deeply care about because models like Claude Code (with Fable running uninterrupted for thirty minutes or more) tend to produce overly defensive, complex, duplicated code that avoids strong invariants and amplifies local fixes; Karpathy noted models are mortally terrified of exceptions. Loops do work well for mechanical or ephemeral tasks—examples cited include reported porting of Bun from Zig to Rust, my own MiniJinja→Go port, performance experiments, security scanning, and LLM-driven experimental workflows judged by simple signals or another LLM—but they’re ill-suited for producing long-lived, deterministic systems where comprehension and invariants matter. #16 📝 OpenAI News How GPT-5 helped immunologist Derya Unutmaz solve a 3-year-old mystery - In late 2025 Derya Unutmaz used GPT‑5 Pro to revisit a 2022 experiment in which early exposure of developing T cells to deoxyglucose (a glucose-like inhibitor) produced far more Th17 inflammatory cells than low-glucose conditions, and GPT‑5 Pro proposed that deoxyglucose interfered with construction of the IL‑2 protein—removing a block on Th17 differentiation and explaining the persistent effect. GPT‑5 Pro also correctly simulated an unpublished experiment showing enhanced lymphoma‑killing by CD8+ T cells, and Unutmaz now uses tools including Codex and GPT‑5.2 Deep Research to compile cancer mutation datasets and draft a T‑cell textbook while emphasizing responsible use per OpenAI’s Preparedness Framework. #17 𝕏 Jason Zhou calls out GLM 5.2’s “insane” pricing at just $1.40 per 1M input tokens and $4.40 per 1M output tokens, making it five times cheaper than Opus. #18 𝕏 Madhu Guru notes that enterprises, agent startups, data providers and labs are still scrambling to define AI business models, moats and value-exchange playbooks in real time. #19 𝕏 Teresa Torres maps event creation to product design—using attendee journey mapping, rapid session prototyping, and feedback-driven iterations—to engineer truly unforgettable experiences (Product at Heart episode). #20 𝕏 Summary: Garry Tan says Linzumi is a multiplayer version of Codex that makes coding truly collaborative for teams. It’s built by Sean Grove, who led OpenAI’s ChatGPT sycophancy-reduction efforts before founding this YC startup. #21 𝕏 Peter Yang observes that human-agent interaction is evolving into managing AI like a high-capability employee—next step: 1-on-1s and performance reviews for Claude. 😅 #22 𝕏 Josh Woodward Florida State University rolled out Google’s NotebookLM on campus, and within weeks students stuck at a C grade completely overhauled their study habits and significantly boosted their grades. #23 𝕏 Philipp Schmid reports that since its Google I/O launch, builders have used @GoogleAIStudio to create over 1,000,000 native Android apps—huge progress with even more to come. #24 𝕏 Logan Kilpatrick reports that in the last month, users created over 1,000,000 native Android apps directly in Google AI Studio. This milestone highlights the platform’s rapid uptake and the breadth of projects being built. #25 𝕏 Jason Zhou finds the nested sub-agent feature in Claude Code needlessly lengthens sessions and worsens context loss, questioning its utility. Found this valuable? Share it with another PM - they can subscribe at genaipm.com Unsubscribe • Switch to Weekly

“The US Department of Commerce banned Anthropic’s safety-enhanced model Fable 5 (and its full-capacity sibling Mythos 5) three days after Fable’s release, forcing Anthropic to pull both models and revert users to Opus 4.8 due to a public jailbreak.”

#4 ▶️ One man just liberated Fable... and now it’s illegal Fireship The US Department of Commerce banned Anthropic’s safety-enhanced model Fable 5 (and its full-capacity sibling Mythos 5) three days after Fable’s release, forcing Anthropic to pull both models and revert users to Opus 4.8 due to a public jailbreak.

“Opus 4.8 replaces Opus 4.7 in Amp's smart mode and solved 62% of tasks in Amp's internal evals (up from 52% for 4.7), running tests and code 15% more per task while making tighter, more focused edits.”

#14 📝 Ampcode Chronicle Opus 4.8 - Opus 4.8 replaces Opus 4.7 in Amp's smart mode and solved 62% of tasks in Amp's internal evals (up from 52% for 4.7), running tests and code 15% more per task while making tighter, more focused edits. It reaches for external tools more appropriately—calling librarian 14 times versus 1 for 4.7 and using edit_file for 79% of file edits (up from 63%)—drops the Read tool, and adds a ~2.5× fast mode that costs 2× base tokens (down from 6× on 4.7).

“LlamaIndex 🦙 rolled out Opus 4.8 with ParseBench results showing gains in tables, semantic formatting, and layout but slight regressions in charts and content faithfulness, alongside a small price/page increase.”

#17 𝕏 LlamaIndex 🦙 rolled out Opus 4.8 with ParseBench results showing gains in tables, semantic formatting, and layout but slight regressions in charts and content faithfulness, alongside a small price/page increase.