本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

Welcome Le Chonk: Mistral Large 4 on Hugging Face

Mistral Large 4 (aka Le Chonk) is a multimodal, multilingual, Mixture-of-Experts (MoE) model from Mistral. It takes text and images as input and generates text. It has about 1T total, 49B active parameter, and a context window of 1M tokens. The model is released in two official variants: FP8 and NVFP4. There are also GGUFs created by Unsloth and the llama.cpp team. It’s distributed under the Mistral Ecosystem Open-Weight License (an MIT derivative that establishes limitations for model-as-a-service offerings). Mistral Large 4 is particularly strong for cybersecurity, coding agents, and multimodality.

Mistral Large 4（又名 Le Chonk）是 Mistral 推出的多模态、多语言混合专家（MoE）模型。它以文本和图像作为输入并生成文本。该模型总参数量约为 1T，激活参数量为 49B，上下文窗口为 1M token。模型发布了两个官方版本：FP8 和 NVFP4。此外，Unsloth 和 llama.cpp 团队也创建了 GGUF 版本。该模型根据 Mistral 生态系统开放权重许可（一种对模型即服务产品有限制的 MIT 衍生许可）进行分发。Mistral Large 4 在网络安全、编码智能体和多模态方面表现尤为出色。

What is new with Mistral Large 4: Mistral Large 4 combines a large pool of experts with sparse attention and a Pixtral vision encoder. The language model selects a small group of experts for each token, while its attention indexer selects which earlier token positions to attend to. These are two different forms of sparsity, where one selects feed-forward networks, the other selects context positions. You can find all the checkpoints in the Mistral Large 4 collection.

Mistral Large 4 的新特性：Mistral Large 4 将庞大的专家池与稀疏注意力机制及 Pixtral 视觉编码器相结合。该语言模型为每个 token 选择一小部分专家，而其注意力索引器则选择需要关注的早期 token 位置。这是两种不同形式的稀疏性，前者选择前馈网络，后者选择上下文位置。你可以在 Mistral Large 4 合集中找到所有的检查点。

Cybersecurity Capabilities: Mistral Large 4 scores 82% on the Artificial Analysis Cyber Index test that asks a model to reproduce a real vulnerability in open-source software and then patch the software. Mistral reports that several closed frontier models score near zero on the same test because they refuse to attempt the work. This is super important for defenders, because fixing a vulnerability usually requires reproducing the vulnerability.

网络安全能力：Mistral Large 4 在 Artificial Analysis Cyber Index 测试中获得了 82% 的分数，该测试要求模型复现开源软件中的真实漏洞并进行修复。Mistral 报告称，几款闭源前沿模型在同一测试中得分接近于零，因为它们拒绝尝试此类任务。这对防御者来说至关重要，因为修复漏洞通常需要先复现漏洞。

Architecture: Mistral Large 4’s text decoder has 72 layers, a hidden dimension of 6,144, and 64 attention heads. Its attention blocks combine Multi-head Latent Attention (MLA) with a DeepSeek Sparse Attention (DSA) indexer. MLA keeps a compressed KV representation, and DSA selects up to 2,048 key positions for each query. For each decoder block, there’s a 3:1 RoPE to NoPE pattern, i.e. 3 decoder layers employ RoPE and every fourth layer uses NoPE.

架构：Mistral Large 4 的文本解码器拥有 72 层，隐藏维度为 6,144，并配备 64 个注意力头。其注意力模块结合了多头潜在注意力（MLA）和 DeepSeek 稀疏注意力（DSA）索引器。MLA 保留压缩的 KV 表示，而 DSA 为每个查询选择最多 2,048 个关键位置。每个解码器块采用 3:1 的 RoPE 与 NoPE 模式，即 3 层解码器使用 RoPE，每第四层使用 NoPE。