Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model

Mage-VL：一种高效的编解码器原生流式多模态基础模型

Standard vision-language models (VLMs) suffer from Moravec’s paradox: they excel at complex offline visual reasoning but struggle with simple streaming perception tasks and process them inefficiently. 标准视觉语言模型（VLMs）深受莫拉维克悖论（Moravec’s paradox）的困扰：它们擅长复杂的离线视觉推理，但在简单的流式感知任务上表现不佳，且处理效率低下。

We present Mage-VL, an efficient codec-native streaming foundation model for real-time multimodal understanding and interaction. At its core, our custom tokenizer, Mage-ViT, replaces uniform frame sampling by selectively encoding dynamic, entropy-rich regions using motion vectors and residual energy across sparse anchor (I) and predicted (P) frames. 我们提出了 Mage-VL，这是一种用于实时多模态理解与交互的高效编解码器原生流式基础模型。其核心在于我们定制的标记器 Mage-ViT，它取代了传统的均匀帧采样，通过利用稀疏锚帧（I帧）和预测帧（P帧）之间的运动向量与残差能量，选择性地编码动态且信息熵丰富的区域。

Operating at a 16 x 16 patch level, this reduces visual token consumption by over 75% while preserving spatiotemporal context. Trained from scratch on approximately 560M unlabeled images and 100M unlabeled video frames, Mage-ViT matches or outperforms flagship encoders trained on billions of image-text pairs. 该模型在 16 x 16 的补丁（patch）级别上运行，在保留时空上下文的同时，将视觉标记（token）的消耗降低了 75% 以上。Mage-ViT 从零开始，在约 5.6 亿张无标签图像和 1 亿帧无标签视频上进行训练，其性能足以媲美甚至超越那些在数十亿图像-文本对上训练的旗舰级编码器。

We establish AI4AI data pipelines encompassing prompt-code joint optimization for multimodal captioning and AI-driven performance diagnosis to guide training recipes. Furthermore, through a bio-inspired dual-system architecture - a lightweight System 1 event gate and a causal System 2 decoder - Mage-VL enables proactive streaming perception. 我们建立了 AI4AI 数据流水线，涵盖了用于多模态字幕生成的提示词-代码联合优化，以及用于指导训练方案的 AI 驱动性能诊断。此外，通过受生物学启发的双系统架构——即轻量级的“系统 1”事件门控和因果“系统 2”解码器——Mage-VL 实现了主动式流式感知。

Extensive evaluations show that Mage-VL-4B matches Qwen3-VL-4B on static tasks while achieving strong gains in video understanding and 2D/3D spatial reasoning, with up to a 3.5x wall-clock inference speedup, and comprehensively surpasses the 15B Phi-4-reasoning-vision baseline. 广泛的评估表明，Mage-VL-4B 在静态任务上与 Qwen3-VL-4B 持平，同时在视频理解和 2D/3D 空间推理方面取得了显著提升，推理速度提升高达 3.5 倍，并全面超越了 15B 参数规模的 Phi-4-reasoning-vision 基准模型。

Beyond model artifacts, we deliver seven key empirical findings covering pre-training data efficiency, variable-resolution scaling, codec system acceleration, VideoQA SFT redundancy, motion-spatial synergy, AI4AI data pipelines, and Zero-Vision SFT for multimodal RL. 除了模型本身，我们还提供了七项关键的实证发现，涵盖了预训练数据效率、可变分辨率缩放、编解码系统加速、视频问答（VideoQA）监督微调（SFT）冗余、运动-空间协同、AI4AI 数据流水线，以及用于多模态强化学习的零视觉（Zero-Vision）SFT。