MiMo V2.6 Flash Local on Two Mac Studios vs Cloud: Benchmark

本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

MiMo V2.6 Flash doesn’t fit on one of our Macs, so we ran it local across two Mac Studios with vision loaded, and it built a payments app that our benchmark scored 0.6605. In Xiaomi’s own cloud, the same model scored 0.7909. I expected worse. Before the benchmark’s 3D ceiling was applied, the local run had earned 0.8309 of the weighted checks and the cloud run 0.838, and the ceiling made the difference.

由于 MiMo V2.6 Flash 无法在我们的单台 Mac 上运行，我们将其部署在两台 Mac Studio 上并加载了视觉功能。它构建了一个支付应用程序，我们的基准测试得分为 0.6605。在小米自家的云端，同一模型的得分为 0.7909。我原本以为结果会更糟。在应用基准测试的 3D 上限之前，本地运行获得了 0.8309 的加权检查分数，而云端运行为 0.838，最终是这个上限造成了差异。

It’s Xiaomi’s MiMo-V2.6-Flash, a 309-billion-parameter mixture of experts with 15 billion active per token. We ran a community MLX conversion, EigenLabs/MiMo-V2.6-Flash-MOPD-MLX-4bit-mtp, and it takes 161 GB on our SSD. Neither of our Mac Studios can hold that alone. Our workhorse is an M3 Ultra with 96 GB and our workmule an M5 Max with 128 GB, so we cut the model’s 48 layers 19 and 29, joined the two Macs with one Thunderbolt 5 cable, and let them pass hidden states to each other over RDMA.

这是小米的 MiMo-V2.6-Flash，一个拥有 3090 亿参数的混合专家模型，每个 token 激活 150 亿参数。我们运行了一个社区版的 MLX 转换模型 EigenLabs/MiMo-V2.6-Flash-MOPD-MLX-4bit-mtp，它在我们的 SSD 上占用 161 GB 空间。我们的两台 Mac Studio 单独都无法容纳它。我们的主力机是 96 GB 内存的 M3 Ultra，辅助机是 128 GB 内存的 M5 Max，因此我们将模型 48 层切分为 19 层和 29 层，用一根雷电 5 线缆连接两台 Mac，并通过 RDMA 让它们相互传递隐藏状态。

Driving it was Goose Swarm, our edition of the open-source goose agent. Why should you care if you’re not about to buy two Mac Studios? Most of what I read about running big models locally stops at whether the weights fit. Making the weights fit took some arithmetic. Keeping two Macs in step, token by token, for five hours took far more of the effort, and so did getting the model to see.

驱动它的是 Goose Swarm，这是我们对开源 goose 代理的改进版本。如果你不打算购买两台 Mac Studio，为什么还要关心这些？我读到的大多数关于在本地运行大模型的文章，都只停留在权重是否装得下这一步。让权重适配需要一些算术计算。而让两台 Mac 在五个小时内逐个 token 地保持同步，所付出的努力要多得多，让模型具备视觉能力也是如此。

It’s one run per setup on one task, so read it as a field report. MiMo V2.6 Flash benchmark: local 0.6605 vs cloud 0.7909. Our Gauntlet benchmark gives an agent 150 model calls to build a payments console: two backend services, a web UI and a 3D field of payment towers, scored on 99 checks. It also has an admission rule. If certain 3D checks fail, the score cannot pass a ceiling, however much credit the rest earned.

每个设置在单一任务上仅运行一次，所以请将其视为一份实地报告。MiMo V2.6 Flash 基准测试：本地 0.6605 对比云端 0.7909。我们的 Gauntlet 基准测试要求代理在 150 次模型调用内构建一个支付控制台：包括两个后端服务、一个 Web UI 和一个 3D 支付塔区域，并根据 99 项检查进行评分。它还有一项准入规则：如果某些 3D 检查失败，无论其余部分获得了多少分数，总分都无法超过一个上限。

The live board is the current word if any of this moves. Both local runs went through goose’s OpenAI-compatible provider pointed at our engine on the workhorse; the cloud run went through OpenRouter, pinned to Xiaomi’s own endpoint. So where did the 0.13 go? Almost all of it is the ceiling. Two 3D checks capped the local run at 0.669.

如果情况有变，实时排行榜是最新参考。两次本地运行都通过 goose 的 OpenAI 兼容提供程序指向我们主力机上的引擎；云端运行则通过 OpenRouter 指向小米自己的端点。那么 0.13 的差距去哪了？几乎全部是因为那个上限。两项 3D 检查将本地运行的分数限制在 0.669。

One, s_tower_geometry (0.778), shoots independent rays at the towers and checks their bevels, ribs and gaps against the spec. The other, t_vs7dbg_truth (0.8), asks the app’s own debug hook which tower sits under a pixel and compares that with what the GPU picking returns; the camera numbers matched, but over empty background the hook said no tower while the GPU pick buffer returned an id (pickTriplet=False). Cloud MiMo passed both.

其中一项 s_tower_geometry (0.778) 向塔楼发射独立射线，并根据规范检查其斜角、肋条和间隙。另一项 t_vs7dbg_truth (0.8) 会询问应用程序自身的调试钩子，确定哪个塔楼位于像素下方，并将其与 GPU 拾取返回的结果进行比较；摄像机编号匹配，但在空白背景上，钩子显示没有塔楼，而 GPU 拾取缓冲区却返回了一个 ID (pickTriplet=False)。云端 MiMo 两项都通过了。

Its own cap of 0.799 came from t_stream_diff, the live-update check, which it scored 0 and the local run scored 1.0. So I read the remaining gap as a handful of 3D details on one page of the app. What we found, in four points: Vision fixed what the text-only run could not. Four checks that scored 0 or 0.5 without a vision tower all scored 1.0 with it, and the old run’s 0.599 ceiling went away.

它自身的 0.799 上限来自 t_stream_diff（实时更新检查），它得分为 0，而本地运行得分为 1.0。因此，我认为剩余的差距只是应用程序页面上的一些 3D 细节问题。我们发现的四点结论：视觉功能修复了纯文本运行无法解决的问题。四项在没有视觉塔楼时得分为 0 或 0.5 的检查，在加入视觉后全部得分为 1.0，旧运行的 0.599 上限也随之消失。

The local run lost 12 checks to the cloud and won 7. Most losses are 3D pixel work; the wins are the live stream and backend recovery (stream diff 1.0 vs 0, notifier exactly-once 1.0 vs 0.69). One 0.7-second race cost three checks and the only penalty. Webhook registration lost to the vendor’s schedule, so a correct forged-signature handler never got tested and one payment never arrived.

本地运行在 12 项检查中输给了云端，但在 7 项中获胜。大部分损失在于 3D 像素处理；获胜点在于实时流和后端恢复（流差异 1.0 对 0，通知器精确一次 1.0 对 0.69）。一次 0.7 秒的竞态条件导致损失了三项检查并受到唯一一次惩罚。Webhook 注册因供应商的时间表而失败，导致正确的伪造签名处理程序从未被测试，且有一笔付款未送达。

The split itself broke twice, in ways a single Mac never would. A dtype mismatch between the Macs and two Macs sampling different tokens each crashed a run. Both are fixed, on a branch. Why the 161 GB MiMo V2.6 Flash MOPD build needs two Macs: The arithmetic is short. EigenLabs lists the weights at 170,974,651,261 bytes, about 159 GiB, in 37 shards. Together our two Macs have 224 GB of unified memory.

拆分本身崩溃了两次，这是单台 Mac 绝不会遇到的情况。两台 Mac 之间的数据类型不匹配以及两台 Mac 采样不同的 token 各导致了一次运行崩溃。两者都在一个分支中得到了修复。为什么 161 GB 的 MiMo V2.6 Flash MOPD 构建需要两台 Mac：算术很简单。EigenLabs 列出的权重为 170,974,651,261 字节，约 159 GiB，分为 37 个分片。我们的两台 Mac 总共拥有 224 GB 的统一内存。

Our engine keeps a 5% reserve under the Metal limit, and the KV cache, the vision tower and the prompt-processing workspace all need room on top of the weights. Despite the name, it isn’t uniformly 4-bit. Its card says the routed experts keep Xiaomi’s upstream MXFP4 values “bit for bit”, while attention, the dense MLP, the embeddings and the output projection are 8-bit affine. Xiaomi’s cloud endpoint is listed on OpenRouter as fp8.

我们的引擎在 Metal 限制下保留了 5% 的余量，而 KV 缓存、视觉塔和提示处理工作区都需要在权重之外占用空间。尽管名字如此，它并非统一的 4-bit。其说明书显示，路由专家保留了小米上游的 MXFP4 值“逐位一致”，而注意力机制、密集 MLP、嵌入和输出投影则是 8-bit 仿射。小米的云端点在 OpenRouter 上被列为 fp8。

What makes 256k possible at all is the architecture. Xiaomi’s card lists 48 layers: 39 use sliding-window attention with a 128-token window and only 9 use global attention. Only those 9 keep a cache that grows with the conversation. Our planner charges 23,040 bytes per token for it across both Macs, plus one copy for the prompt cache and a workspace for the attention scores.

之所以能实现 256k，全靠其架构。小米的说明书列出了 48 层：39 层使用 128 个 token 窗口的滑动窗口注意力机制，只有 9 层使用全局注意力。只有这 9 层保留了随对话增长的缓存。我们的规划器在两台 Mac 上为每个 token 分配 23,040 字节，外加一份提示缓存副本和注意力分数工作区。

At 262,144 tokens the plan said 75.1 GB on the workhorse and 107.05 GB on the workmule. Measured with a 210,724-token prompt, the peaks were 75.9 GB and 109.1 GB, and swap stayed flat on both Macs. Why 19 layers on the workhorse and 29 on the workmule? An M3 Ultra with less memory, which also carries the vision tower, gets fewer layers. It also gets the last layers: mlx-lm’s pipeline code “split[s] layers in reverse so rank=0 gets the last layers”, and the workhorse is rank 0.

在 262,144 个 token 时，计划显示主力机占用 75.1 GB，辅助机占用 107.05 GB。使用 210,724 个 token 的提示进行测量时，峰值分别为 75.9 GB 和 109.1 GB，两台 Mac 的交换空间保持平稳。为什么主力机 19 层而辅助机 29 层？内存较小的 M3 Ultra 还要承载视觉塔，因此分配的层数较少。它还负责最后几层：mlx-lm 的流水线代码“反向拆分层，以便 rank=0 获得最后几层”，而主力机就是 rank 0。

There is one copy of the weights. It lives on the workhorse’s external SSD and the workmule reads it over NFS, so a cold load took 241 to 246 seconds across five loads with no NFS errors. That -mtp in the build’s name means it also ships multi-token-prediction heads. Our split’s loader drops them, so the speeds below are plain decoding. Splitting it across two Mac Studios over Thunderbolt 5 RDMA (MLX JACCL) Apple added RDMA over Thunderbolt in macOS 26.2. Its release notes describe it as low-latency comm.

权重只有一份。它存储在主力机的外置 SSD 上，辅助机通过 NFS 读取，因此在五次冷加载中，加载时间为 241 到 246 秒，且没有出现 NFS 错误。构建名称中的 -mtp 意味着它还附带多 token 预测头。我们拆分器的加载程序丢弃了它们，因此下方的速度是纯解码速度。通过雷电 5 RDMA (MLX JACCL) 在两台 Mac Studio 之间拆分：苹果在 macOS 26.2 中增加了雷电 RDMA 支持。其发行说明将其描述为低延迟通信。