Welcome Muse Spark Open: open-weight frontier multimodal mixture-of-experts model from Meta

本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

Back to Articles Welcome Muse Spark Open: open-weight frontier multimodal mixture-of-experts model from Meta Published October 15, 2026 Update on GitHub Upvote 24 +18 Pedro Cuenca pcuenq Follow merve merve Follow ben burtenshaw burtenshaw Follow Guarin Flück guarin Follow Quentin Gallouédec qgallouedec Follow Alvaro Bartolome alvarobartt Follow Serveurperso Serveurperso Follow Jun Kim Jundot Follow Muse Spark Open is an open-weight model from Meta. The mixture of experts model brings text, images, audio, and video into a model with approximately 816 billion parameters, around 42 billion active per token, and a context window of up to one million tokens. It generates text, giving developers a common interface for asking questions about different kinds of input. Following Muse Glimmer, Muse Spark Open introduces a mixture-of-experts architecture at a much larger scale, and the most capable US open-weights model to date. Meta trained Muse Spark Open for agentic coding: reading a codebase, writing and debugging code, and calling tools. It also takes images, audio, and video as input, so the same model can read a screenshot, listen to a voice note, or watch a screen recording while it works on your code. This post shows you how to run it, how to fine-tune it, and what we built with it. Find all the models here.

欢迎阅读文章：Muse Spark Open：来自 Meta 的开放权重前沿多模态混合专家模型。发布于 2026 年 10 月 15 日。Muse Spark Open 是 Meta 推出的开放权重模型。该混合专家模型将文本、图像、音频和视频整合到一个模型中，拥有约 8160 亿参数，每个 token 激活约 420 亿参数，上下文窗口高达 100 万 token。它能够生成文本，为开发者提供了一个针对不同输入类型进行提问的通用接口。继 Muse Glimmer 之后，Muse Spark Open 以更大的规模引入了混合专家架构，是迄今为止能力最强的美国开放权重模型。Meta 对 Muse Spark Open 进行了代理式编码训练：阅读代码库、编写和调试代码以及调用工具。它还支持图像、音频和视频输入，因此同一个模型可以在处理代码时读取截图、收听语音笔记或观看屏幕录制。本文将展示如何运行、微调该模型以及我们用它构建的内容。

Overall Capabilities and Architecture Meta describes Muse Spark Open as a coding-optimized model trained on real developer workflows such as multi-file refactors and long debugging sessions. Muse Spark Open is an agentic focused model that plans, acts, observes, and self-corrects over long tasks, including browsing the web with images.

整体能力与架构：Meta 将 Muse Spark Open 描述为一种针对编码优化的模型，它在真实开发者工作流（如多文件重构和长时间调试会话）上进行了训练。Muse Spark Open 是一款专注于代理任务的模型，能够在长任务中进行规划、行动、观察和自我纠正，包括通过图像浏览网页。

Feature Muse Spark Open Model size ~816B total parameters; ~42B active per token Expert routing 256 experts per MoE layer; 8 selected per token Inputs Text, images, audio, and video Output Text Context window Up to 1M tokens Checkpoint formats BF16 and NVFP4 The checkpoint repositories are Muse-Spark-Open-816B-A42B and Muse-Spark-Open-816B-A42B-NVFP4. Muse Spark Open is the full-size model of the Muse series of models. Muse Glimmer, which Meta released in August, is a small dense model from the same series, ideal for running locally. Both models use the same tokenizer and vocabulary. The chat template follows the Muse Glimmer template, with additions for the new modalities. In Muse Spark Open the input embedding and the output layer are separate matrices of about 1.66B parameters each.

特性：Muse Spark Open。模型规模：总参数约 8160 亿；每个 token 激活约 420 亿。专家路由：每层 MoE 包含 256 个专家；每个 token 选择 8 个。输入：文本、图像、音频和视频。输出：文本。上下文窗口：高达 100 万 token。检查点格式：BF16 和 NVFP4。检查点仓库为 Muse-Spark-Open-816B-A42B 和 Muse-Spark-Open-816B-A42B-NVFP4。Muse Spark Open 是 Muse 系列模型的全尺寸版本。Meta 在 8 月发布的 Muse Glimmer 是该系列中的小型密集模型，非常适合本地运行。两个模型使用相同的分词器和词汇表。聊天模板沿用了 Muse Glimmer 模板，并针对新模态进行了补充。在 Muse Spark Open 中，输入嵌入层和输出层是各自独立的矩阵，每个约 16.6 亿参数。

Mixture of Experts The language model has 62 layers and a hidden dimension of 8192. Each layer has 128 query heads and 16 key-value heads (grouped-query attention). Each head has 64 dimensions. The attention block has an output gate and an output normalization layer. The model also applies normalization to the queries and the keys (QK-norm). Each block combines its input and its output through a learned residual gate, not through a plain sum. Muse Glimmer uses one dense feed-forward network in each layer. Muse Spark Open replaces this dense network with a Mixture-of-Experts (MoE) block. Each MoE block has 256 experts, and a router selects 8 experts for each token. Each expert is a feed-forward network that projects from 8192 to 4096 dimensions and back to 8192, then applies its own normalization to its output. The experts hold ~97% of the total model parameters. Only 8 experts run per token, so the model computes each token with about 42B parameters. All 256 experts stay in memory, so deployment needs the memory of an 816B-parameter model, but runs at the much faster speed of a 42B model. The BF16 checkpoint is about 1.5 TB in safetensors format. The NVFP4 checkpoint stores the parameters in 4-bit precision with negligible loss in accuracy, specifically for NVIDIA Blackwell GPUs. In transformers, the BF16 checkpoint runs across 4 nodes of 8 H200 GPUs with tensor parallelism (TP=32) and the NVFP4 runs on a single B200 node. If you want to deploy Muse Spark Open, check out the llama.cpp, vLLM, and SGLang recipes below.

混合专家：该语言模型拥有 62 层，隐藏维度为 8192。每一层有 128 个查询头和 16 个键值头（分组查询注意力）。每个头有 64 个维度。注意力模块包含一个输出门和一个输出归一化层。模型还对查询和键应用了归一化（QK-norm）。每个模块通过学习到的残差门结合输入和输出，而不是简单的相加。Muse Glimmer 在每一层使用一个密集前馈网络，而 Muse Spark Open 将其替换为混合专家（MoE）模块。每个 MoE 模块有 256 个专家，路由器为每个 token 选择 8 个专家。每个专家都是一个前馈网络，将维度从 8192 投影到 4096 再回到 8192，然后对其输出应用归一化。专家占据了总模型参数的约 97%。每个 token 仅运行 8 个专家，因此模型计算每个 token 时使用约 420 亿参数。所有 256 个专家都驻留在内存中，因此部署需要 8160 亿参数模型的内存容量，但运行速度却快得多，相当于 420 亿参数模型。BF16 检查点在 safetensors 格式下约为 1.5 TB。NVFP4 检查点以 4 位精度存储参数，精度损失可忽略不计，专为 NVIDIA Blackwell GPU 设计。在 transformers 中，BF16 检查点运行在 4 个节点（每个节点 8 个 H200 GPU）上，采用张量并行（TP=32），而 NVFP4 运行在单个 B200 节点上。如需部署 Muse Spark Open，请查看下方的 llama.cpp、vLLM 和 SGLang 配方。

Perception Encoder A perception encoder converts images and video frames into tokens for the language model, just like Muse Glimmer. The encoder has 1.8B parameters, 50 layers, a hidden dimension of 1536, and 16 attention heads. It splits each image into 14×14 patches and produces features with 4096 dimensions. Position information comes from learned absolute embeddings and 2D RoPE. The feed-forward network in each layer uses LayerNorm and GELU. Every fourth layer and the last layer use full attention, and the other layers use sparse attention. Muse Glimmer 30B uses an encoder with the same architecture. The design follows the Perception Encoder work from Meta. Video input goes through the same encoder as images.

感知编码器：感知编码器将图像和视频帧转换为语言模型的 token，与 Muse Glimmer 一样。该编码器拥有 18 亿参数、50 层、1536 的隐藏维度和 16 个注意力头。它将每张图像分割成 14×14 的补丁，并生成 4096 维的特征。位置信息来自学习到的绝对嵌入和 2D RoPE。每一层的前馈网络使用 LayerNorm 和 GELU。每隔四层以及最后一层使用全注意力机制，其他层使用稀疏注意力。Muse Glimmer 30B 使用了相同架构的编码器。该设计遵循 Meta 的感知编码器研究。视频输入通过与图像相同的编码器进行处理。

Audio Encoder Muse Spark Open accepts audio as input through its audio encoder. An audio tokenizer converts the audio into tokens, and the chat template marks the audio position with <|audio|> tokens. The encoder emits one audio token every 80 ms, which yields 12.5 tokens per second. So let’s say, you have an audio of 10 seconds, the number of input audio tokens would be 125. The encoder itself is a convolutional pyramid where each stage consists of a patch embedder of different patch sizes and 2 convolutional blocks of 1d kernels of size 7 with stride of 1. These stages are repeated for patch sizes of [16, 6, 5, 4]. At the end of the final stage, the temporal frames are fed to a final block of RMSNorm → linear projection → RMSNorm.

音频编码器：Muse Spark Open 通过其音频编码器接收音频输入。音频分词器将音频转换为 token，聊天模板使用 <|audio|> token 标记音频位置。编码器每 80 毫秒发出一个音频 token，即每秒 12.5 个 token。假设你有 10 秒的音频，输入音频 token 的数量将是 125 个。编码器本身是一个卷积金字塔，每一级由不同补丁大小的补丁嵌入器和 2 个 1D 卷积块（核大小为 7，步长为 1）组成。这些阶段针对 [16, 6, 5, 4] 的补丁大小进行重复。在最后阶段结束时，时间帧被馈送到一个最终模块：RMSNorm → 线性投影 → RMSNorm。

Long Context The context window is 1M tokens at inference. The configuration sets max_position_embeddings to 1,048,576. This includes image, video, and audio tokens in the same window as text. Grouped-query attention keeps the KV cache small for a model of this size. Each token adds 2 × 16 heads × 64 dimensions × 62 layers = 126,976 values to the cache. In BF16, this is about 250 KB per token and about 266 GB for a full 1M-token context. Inference Support Transformers The Muse Spark Open implementation provides MuseSparkFo

长上下文：推理时的上下文窗口为 100 万 token。配置将 max_position_embeddings 设置为 1,048,576。这包括与文本处于同一窗口中的图像、视频和音频 token。分组查询注意力机制使该规模模型的 KV 缓存保持在较小水平。每个 token 向缓存添加 2 × 16 个头 × 64 个维度 × 62 层 = 126,976 个值。在 BF16 格式下，每个 token 约为 250 KB，完整的 100 万 token 上下文约为 266 GB。推理支持 Transformers：Muse Spark Open 的实现提供了 MuseSparkFo