Meta is back with Muse Glimmer: local, agentic, multimodal, and open source

Meta 重磅回归：Muse Glimmer 正式发布，具备本地化、智能体、多模态及开源特性

Great news from the OGs of open source LLMs! Muse Glimmer, released today, is Meta’s new multimodal model, especially designed for local agentic use cases. Distilled from Muse to 30B parameters, and released under the Apache 2.0 license, it’s ideal deploying locally for privacy, reducing costs, or just hacking around. It’s intended for privacy-aware applications such as coding, document analysis, personal assistants, Claw- or Hermes-like setups.

来自开源大模型先驱的好消息！Meta 今日发布了全新的多模态模型 Muse Glimmer，专为本地智能体（Agentic）应用场景设计。该模型由 Muse 蒸馏至 300 亿参数，并采用 Apache 2.0 协议开源，非常适合本地部署以保障隐私、降低成本或进行技术探索。它旨在服务于注重隐私的应用，如编程辅助、文档分析、个人助理以及类似 Claw 或 Hermes 的系统架构。

To celebrate, we are shipping with Meta day-0 support in transformers, llama.cpp, vLLM, Inference Endpoints, and other libraries. We built a few cool things and explain our findings in this blog. Check out the demos below for inspiration. You can find Muse Glimmer on the Hugging Face Hub.

为庆祝发布，我们已在 transformers、llama.cpp、vLLM、Inference Endpoints 及其他库中提供对 Meta 的首日支持。我们构建了一些有趣的应用，并在本博客中分享了我们的发现。请查看下方的演示以获取灵感。你可以在 Hugging Face Hub 上找到 Muse Glimmer。

Architecture

架构

Muse Glimmer is a dense 30B parameter model consisting of: 2B ViT-style encoder for vision (Perception Encoder) and a 28B parameter text decoder. In addition to the main VLM, there’s also a speculative decoding drafter implemented on DFlash. Usage of this module is optional, and it can provide much faster generation in exchange for some memory cost. We found this drafter to be particularly well suited to structured content generation such as coding.

Muse Glimmer 是一个 300 亿参数的稠密模型，包含：一个 20 亿参数的视觉 ViT 风格编码器（Perception Encoder）和一个 280 亿参数的文本解码器。除了主视觉语言模型（VLM）外，还有一个基于 DFlash 实现的投机解码草稿模型（Speculative Decoding Drafter）。该模块可选择性使用，能在增加少量内存开销的情况下显著提升生成速度。我们发现该草稿模型特别适合编程等结构化内容的生成。

Text Decoder

文本解码器

The language model uses the following architecture components: Hybrid attention: Alternating between three sliding window layers (of 2,048 tokens) using rotary position embedding, followed by a fourth layer that uses full attention and NoPE (no positional embedding). The pattern is therefore (SWA, SWA, SWA, Full), repeated 13 times to a total of 52 layers. This allows the model to retain relative order and distance information with RoPE and preserve information globally with NoPE.

该语言模型采用了以下架构组件：混合注意力机制（Hybrid attention）：在三个使用旋转位置编码（RoPE）的滑动窗口层（2,048 tokens）与一个使用全注意力机制且无位置编码（NoPE）的层之间交替。其模式为（SWA, SWA, SWA, Full），重复 13 次，共计 52 层。这使得模型既能通过 RoPE 保留相对顺序和距离信息，又能通过 NoPE 在全局范围内保存信息。

Gated Grouped-Query Attention: Each key-value head is shared by 16 query heads, which reduces KV-cache memory by 16x and makes generation faster and cheaper. Q-K normalization with extra query scaling: Before computing attention, Muse Glimmer applies RMS normalization to every query and key head to keep attention logits stable. After this, queries are multiplied by a scale factor to set the target logit scale after normalization. The extra query scaling behaves like an inverse temperature at the softmax level.

门控分组查询注意力（Gated Grouped-Query Attention）：每个键值（KV）头由 16 个查询头共享，这减少了 16 倍的 KV 缓存内存占用，使生成过程更快、成本更低。带有额外查询缩放的 Q-K 归一化：在计算注意力之前，Muse Glimmer 对每个查询和键头应用 RMS 归一化，以保持注意力 Logits 的稳定性。此后，查询会乘以一个缩放因子，以设定归一化后的目标 Logit 尺度。这种额外的查询缩放类似于 Softmax 层级的反向温度调节。

Perception Encoder

感知编码器

Muse Glimmer uses one image encoder to handle both images and videos. Unlike the relatively small vision encoders used in other VLMs, this is a sizable 2B ViT-like model designed after the Perception Encoder architecture. Perception Encoder was previously introduced by Meta as a backbone for various downstream spatial and multimodal tasks.

Muse Glimmer 使用单一图像编码器来处理图像和视频。与其它 VLM 中使用的较小视觉编码器不同，这是一个基于 Perception Encoder 架构设计的 20 亿参数大型 ViT 模型。Perception Encoder 此前由 Meta 推出，作为各种下游空间和多模态任务的骨干网络。

Transformers

Transformers 库支持

Upgrade transformers to the latest version to be able to use Muse Glimmer. pip install --upgrade transformers accelerate Muse Glimmer comes with day-0 support in transformers, both for the main model and the speculative decoding drafter. You can use AutoModelForMultimodal.

请将 transformers 升级到最新版本以使用 Muse Glimmer。 pip install --upgrade transformers accelerate Muse Glimmer 在 transformers 中提供首日支持，涵盖主模型和投机解码草稿模型。你可以使用 AutoModelForMultimodal 进行调用。