Gemma 4 Technical Report

Computer Science > Computation and Language arXiv:2607.02770 (cs) [Submitted on 2 Jul 2026] 计算机科学 > 计算与语言 arXiv:2607.02770 (cs) [提交于 2026 年 7 月 2 日]

Title: Gemma 4 Technical Report 标题：Gemma 4 技术报告

Abstract: 摘要：

We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. 我们推出了 Gemma 4，这是 Gemma 模型家族中新一代开放权重、原生多模态语言模型。

Designed to advance compute efficiency and reasoning, the Gemma 4 model suite features dense and Mixture-of-Experts architectures, ranging from 2.3B to 31B parameters. Gemma 4 模型套件旨在提升计算效率和推理能力，采用了密集（dense）和混合专家（Mixture-of-Experts）架构，参数规模从 23 亿到 310 亿不等。

Alongside improved vision and audio encoders for all model sizes, we propose a unified, encoder-free architecture for our 12B model, which ingests raw audio and image patches. 除了为所有模型尺寸改进了视觉和音频编码器外，我们还为 12B 模型提出了一种统一的、无编码器架构，该架构可以直接摄取原始音频和图像补丁（patches）。

Furthermore, we integrate a thinking mode, enabling Gemma models to generate reasoning traces prior to responding. 此外，我们集成了“思考模式”（thinking mode），使 Gemma 模型能够在回答之前生成推理轨迹。

We improve inference speed, memory, and compute efficiency, as well as long-context abilities through critical design choices. 通过关键的设计选择，我们提升了推理速度、内存和计算效率，以及长上下文处理能力。

Gemma 4 establishes a leap in performance across STEM, multimodal, and long-context benchmarks, and rivals larger, frontier open models in human-rated tasks. Gemma 4 在 STEM（科学、技术、工程和数学）、多模态和长上下文基准测试中实现了性能飞跃，并在人类评估任务中可与更大规模的前沿开源模型相媲美。