GenAI PM
tool11 mentions· Updated Jun 23, 2026

Gemini Interactions API

Google’s interactions-oriented API for model and agent workflows. The newsletter notes it reaching GA and being available as an npm skill.

Key Highlights

  • Gemini Interactions API evolved from an agent-building interface into a generally available Google API for structured model and tool workflows.
  • Key capabilities highlighted in coverage include multimodal function calling, long-video understanding, streaming, and step-based interaction design.
  • The API shifted from simple user/model roles to explicit workflow steps, making agent behavior easier to structure and manage.
  • Supporting guides, TypeScript frameworks, and npm skill packaging lower the barrier for teams prototyping Gemini-based agents.
  • For AI PMs, the API is especially relevant for planning multimodal products, operationalizing agent flows, and speeding developer onboarding.

Gemini Interactions API

Overview

Gemini Interactions API is Google’s interactions-oriented API for building model-driven and agentic workflows with Gemini. Rather than centering development around a simple prompt/response pattern, it frames application behavior as a sequence of structured interaction steps such as user input, tool calls, thoughts, and model outputs. Across the newsletter coverage, it appears as a key interface for building autonomous agents, multimodal applications, streaming experiences, and long-context workflows on top of Gemini models.

For AI Product Managers, the Gemini Interactions API matters because it signals a more production-oriented abstraction for agent products. The mentions point to practical capabilities that PMs care about: multimodal function calling, streaming, stateful and stateless execution patterns, context management, model switching, and packaged developer skills that accelerate onboarding. Its progression to general availability also suggests increased maturity for teams considering Google’s ecosystem for agentic products.

Key Developments

  • 2026-02-14: Gemini Interactions API added multimodal function calling, enabling agents to process and return real images alongside text with Gemini 3 image capabilities.
  • 2026-02-17: Philipp Schmid introduced a minimal TypeScript agent framework for the API, split into `agents-core` for the lightweight execution loop and `agent` for higher-level tools, hooks, sessions, skills, and subagents.
  • 2026-03-05: A Gemini Interactions API skill launched for building advanced agentic apps, installable globally through Vercel or Context7AI CLIs.
  • 2026-03-10: The API was highlighted for understanding minutes to hours of YouTube video in seconds with a single API call, showcasing strong long-video and multimodal analysis capabilities.
  • 2026-03-17: Philipp Schmid published a Nano Banana 2 guide using the Gemini Interactions API across text-to-image generation, web search grounding, image search, and reference-based image workflows.
  • 2026-05-08: The API interaction model shifted from rigid `user`/`model` roles to discrete steps like `user_input`, `thought`, `function_call`, `tool_call`, and `model_output`, while also consolidating response format controls.
  • 2026-05-13: New `thought` steps and encrypted signatures were documented, along with stateful vs. stateless modes, easier model switching, and context management patterns for agent development.
  • 2026-05-19: A new streaming guide was released to simplify building real-time streaming applications with the Gemini Interactions API.
  • 2026-06-12: Philipp Schmid rewrote the getting-started guide, laying out 11 steps from a first API call to autonomous agents.
  • 2026-06-23: Gemini Interactions API reached general availability and became available as an npm skill (`gemini-interactions-api`) for injecting current SDK patterns, baked-in rules, and updated model versions.

Relevance to AI PMs

1. Designing agent products: The API’s step-based interaction model gives PMs a more concrete way to spec agent workflows, including where user input, reasoning traces, tool execution, and final outputs should appear in the product experience. 2. Planning multimodal use cases: Newsletter mentions show support for images, video understanding, search grounding, and mixed text/image outputs, which helps PMs evaluate new product surfaces beyond chat-only experiences. 3. Reducing time-to-prototype and onboarding risk: The getting-started guides, npm skill packaging, and TypeScript agent frameworks suggest a faster path for engineering teams to prototype and standardize agent implementations.

Related

  • Google / Google Gemini / Gemini: The broader model and platform ecosystem that the Interactions API sits within.
  • Deep Research API / deep-research: An alias in coverage that suggests overlap with research-oriented or agentic workflows built on Gemini.
  • nano-banana-2: A related model/workflow example used with the API for multimodal image generation and grounding use cases.
  • agents-core / agent: Lightweight TypeScript framework components created to make Gemini Interactions API agent loops, tools, sessions, and subagents easier to implement.
  • philipp-schmid / phil-schmid: The most frequently cited builder and educator in the newsletter coverage, responsible for multiple guides, skills, and framework examples.
  • logan-kilpatrick: Mentioned alongside Gemini API cost-control guidance, relevant for PMs thinking about operational limits and spend management.
  • gemini-3-deep-think: A related Gemini model/entity that may connect to advanced reasoning and agent workflows in the broader ecosystem.

Newsletter Mentions (11)

2026-06-23
“Philipp Schmid announces the Gemini Interactions API is now GA and available as an npm skill (`npx skills add google-gemini/gemini-skills --skill gemini-interactions-api --global`) to inject baked-in rules, SDK patterns, and current model versions.”

The item says the Gemini Interactions API is now generally available and packaged as an npm skill.

2026-06-12
“#11 𝕏 Philipp Schmid rewrote the Gemini Interactions API getting-started guide from scratch, outlining 11 steps to go from your first API call to running autonomous agents.”

#11 𝕏 Philipp Schmid rewrote the Gemini Interactions API getting-started guide from scratch, outlining 11 steps to go from your first API call to running autonomous agents.

2026-05-19
“Philipp Schmid published a new guide for streaming in the Gemini Interactions API to make building streaming applications super easy.”

#12 𝕏 Philipp Schmid published a new guide for streaming in the Gemini Interactions API to make building streaming applications super easy. Just point your agent to it and let it handle the rest.

2026-05-13
“#7 𝕏 Philipp Schmid published a guide for Gemini Interactions API’s new `thought` steps and encrypted signatures, detailing stateful vs. stateless modes, seamless model switching, and effortless context management to supercharge agent development.”

#7 𝕏 Philipp Schmid published a guide for Gemini Interactions API’s new `thought` steps and encrypted signatures, detailing stateful vs. stateless modes, seamless model switching, and effortless context management to supercharge agent development. Also covered by: @Sundar Pichai

2026-05-08
“Philipp Schmid updated the Gemini Interactions API to replace rigid `user`/`model` roles with discrete “steps” (user_input, thought, function_call, tool_call, model_output, etc.), consolidated response_format controls, and added a toggle in the docs.”

The newsletter highlights a structural change to Gemini’s interaction model for developers.

2026-03-17
“#7 𝕏 Philipp Schmid wrote a developer guide for Nano Banana 2 with the Gemini Interactions API, walking through four use cases: text-to-image photorealistic Kyoto travel poster generation, Web Search grounding with real landmark facts, Image Search for accurate photos, and referen...”

Today's top 25 insights for PM Builders, ranked by relevance from Blogs, X, YouTube, and LinkedIn. #7 𝕏 Philipp Schmid wrote a developer guide for Nano Banana 2 with the Gemini Interactions API, walking through four use cases: text-to-image photorealistic Kyoto travel poster generation, Web Search grounding with real landmark facts, Image Search for accurate photos, and referen... #8 𝕏 Logan Kilpatrick explains that the Gemini API offers two cost-control levers—global billing account caps to cap overall spend and user-set spend caps to limit individual usage—detailing how each works to manage billing.

2026-03-10
“#10 𝕏 Philipp Schmid shows how the Gemini Interactions API can process minutes to hours of YouTube video content in seconds with a single API call, highlighting a major leap in video understanding.”

The newsletter presents the API as a breakthrough for processing and understanding long video content. It is framed as a practical capability for builders working with multimodal data.

2026-03-05
“Philipp Schmid launched a new Gemini Interactions API skill for building advanced agentic apps with Gemini models, installable globally via the Vercel or Context7AI CLIs.”

#6 𝕏 Philipp Schmid launched a new Gemini Interactions API skill for building advanced agentic apps with Gemini models, installable globally via the Vercel or Context7AI CLIs.

2026-02-17
“Philipp Schmid built a minimal TypeScript agent framework for the Gemini Interactions API, split into agents-core (~500 LOC for a clean loop, streaming events and tool calling) and agent (built-in tools, hooks, sessions, skills & subagents).”

#3 𝕏 Philipp Schmid built a minimal TypeScript agent framework for the Gemini Interactions API, split into agents-core (~500 LOC for a clean loop, streaming events and tool calling) and agent (built-in tools, hooks, sessions, skills & subagents). #4 ▶️ Claude Code built me a $273/Day online directory Greg Isenberg Frey Chu uses Claude Code, Outscraper, Crawl for AI and Claude Vision to automate scraping, cleaning and enriching 71,000 Google Maps entries into a luxury restroom trailers directory of 725 listings in four days for under $250.

2026-02-14
“Philipp Schmid announced the Gemini Interactions API now supports multimodal function calling, letting agents natively see, process, and return real images (not just text) with Gemini 3’s image processing and mixed text/image outputs.”

#2 𝕏 Philipp Schmid announced the Gemini Interactions API now supports multimodal function calling, letting agents natively see, process, and return real images (not just text) with Gemini 3’s image processing and mixed text/image outputs. Also covered by: @Jeff Dean

Related

Philipp Schmidperson

An AI practitioner who shared information about an MCP public roadmap. He is mentioned as the source of protocol-related developments.

Logan Kilpatrickperson

Google AI product leader frequently cited for developer-tool updates. Here he is associated with Google AI Studio and GitHub integration announcements.

Geminitool

Google’s AI model family and product layer referenced as powering Pixel 11 experiences and API integrations. PMs should see it as a central Google AI platform spanning consumer and developer use cases.

Googlecompany

A major AI company referenced throughout the newsletter in relation to Gemini, Notebook, Pixel integrations, and WeatherNext 2. It is associated here with the open-sourcing of Credentio and other product updates.

Nano Banana 2tool

An image-generation model or capability used in Google Earth on the web. It enables prompt-based visual reimagining of locations with satellite and 3D imagery.

Phil Schmidperson

AI product and developer advocate who shares predictions on generative AI trends. Relevant for AI PMs tracking market direction and product strategy.

Deep Researchconcept

A research capability embedded into Perplexity Computer as a built-in skill. For PMs, it indicates the packaging of advanced research into agent workflows.

Google Geminitool

Google's Gemini model family referenced in guidance for integrating it into Android apps.

Stay updated on Gemini Interactions API

Get curated AI PM insights delivered daily — covering this and 1,000+ other sources.

Subscribe Free