Introducing agentic video understanding with Gemini

隆重推出 Gemini 代理式视频理解功能

Today, we’re launching agentic video understanding across our latest models: Gemini 3.7 Flash, 3.6 Flash and 3.5 Flash-Lite. This new capability improves accuracy while dramatically reducing token usage and costs for video analysis. 今天，我们在最新的模型 Gemini 3.7 Flash、3.6 Flash 和 3.5 Flash-Lite 上推出了代理式视频理解（agentic video understanding）功能。这项新能力在提高准确性的同时，大幅降低了视频分析的 Token 消耗和成本。

Similar to agentic vision, which combines code execution with Gemini models’ native image understanding, agentic video understanding uses Gemini’s native video tools to improve performance and unlock new capabilities for video processing like sub-second moment retrieval, more accurate anomaly detection, precise counting and more. 类似于将代码执行与 Gemini 模型原生图像理解相结合的“代理式视觉”（agentic vision），代理式视频理解利用 Gemini 的原生视频工具来提升性能，并解锁视频处理的新能力，例如亚秒级时刻检索、更准确的异常检测、精确计数等。

The feature is available today for video uploads and YouTube videos via the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform. 该功能即日起通过 Google AI Studio 和 Gemini 企业代理平台（Gemini Enterprise Agent Platform）中的 Gemini API 提供，支持上传视频和 YouTube 视频。

Benchmarks

基准测试

Unlike current ‘static’ processing, where the model ingests the video at a fixed frames-per-second rate (default 1 FPS, adjustable via API), agentic video understanding pairs the model’s core reasoning with native video tools to dynamically search, scan, and inspect target video segments across visual frames, audio, and transcripts. 与当前模型以固定帧率（默认为 1 FPS，可通过 API 调整）摄取视频的“静态”处理方式不同，代理式视频理解将模型的核心推理能力与原生视频工具相结合，能够跨视觉帧、音频和转录内容动态地搜索、扫描和检查目标视频片段。

Across standard video analysis benchmarks, Gemini models with agentic video understanding reduce analysis costs by up to 66% and token consumption by up to 88%, while improving accuracy by up to 7%. 在标准的视频分析基准测试中，使用代理式视频理解的 Gemini 模型将分析成本降低了高达 66%，Token 消耗降低了高达 88%，同时准确率提升了高达 7%。

These efficiency gains are especially pronounced on long-form video (from 10-minute how-to guides to 90-minute lectures and multi-hour recordings), where static processing forces developers to choose between high token costs or techniques that drop critical details. 这些效率提升在长视频（从 10 分钟的操作指南到 90 分钟的讲座以及数小时的录像）中尤为显著，在这些场景下，静态处理迫使开发者必须在“高昂的 Token 成本”与“丢失关键细节的技术”之间做出选择。

How it works

工作原理

Instead of static processing where the model ingests media streams at a fixed frame rate, agentic video understanding enables Gemini to take an active, goal-directed role in determining what to watch, at what speed, and through which modality (frames, audio, or transcript), fetching only the moments and signals needed. 代理式视频理解不再采用模型以固定帧率摄取媒体流的静态处理方式，而是使 Gemini 能够发挥主动的、目标导向的作用，自主决定观看什么、以什么速度观看，以及通过哪种模态（帧、音频或转录）进行观看，仅获取所需的时刻和信号。

While developers could previously do this manually, with agentic video understanding, Gemini can accomplish it through an agentic loop, invoking an internal tool to load the relevant part of the video file, significantly reducing development overheads. 虽然开发者以前可以手动完成这些操作，但有了代理式视频理解，Gemini 可以通过代理循环（agentic loop）自动完成，调用内部工具加载视频文件的相关部分，从而显著降低开发成本。

Capabilities and use cases

功能与应用场景

Agentic video understanding transforms how developers can process long-form video content across a variety of demanding applications. 代理式视频理解改变了开发者处理长视频内容的方式，适用于各种高要求应用：

Sub-second moment retrieval: Pinpoint split-second state changes and tight cut boundaries that are easily missed at 1 FPS, making precise automated video editing possible. 亚秒级时刻检索： 精准定位在 1 FPS 下容易错过的瞬间状态变化和紧凑的剪辑边界，使精确的自动化视频编辑成为可能。
Long-form needle-in-a-haystack search: Answer complex queries across multi-hour videos without consuming millions of tokens. 长视频“大海捞针”式搜索： 在数小时的视频中回答复杂查询，而无需消耗数百万个 Token。
Anomaly detection: Resample interesting time windows at higher FPS to inspect rapid motion and subtle visual artifacts. 异常检测： 以更高的帧率对感兴趣的时间窗口进行重采样，以检查快速运动和细微的视觉伪影。
Counting action & object: Accurately track repeated physical movements and distinct objects over time. 动作与物体计数： 随时间推移准确跟踪重复的物理动作和特定物体。

Getting started

如何开始

Agentic video understanding is available via the Gemini API in Google AI Studio and Gemini Enterprise Agent Platform, launching across Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite. It uses standard Gemini API token pricing with no additional feature fee. 代理式视频理解现已通过 Google AI Studio 和 Gemini 企业代理平台中的 Gemini API 提供，支持 Gemini 3.7 Flash、3.6 Flash 和 3.5 Flash-Lite。它采用标准的 Gemini API Token 定价，无需额外功能费用。

To enable it, simply set processing to “agentic” in the API configuration. Read our developer guide to get more insights into the feature and how to get started. 要启用该功能，只需在 API 配置中将 processing 设置为 "agentic"。请阅读我们的开发者指南，以深入了解该功能及如何开始使用。