Apple Silicon and macOS VMs: 11–16× Faster LLM Inference with llama.cpp

Apple Silicon 与 macOS 虚拟机：利用 llama.cpp 实现 11–16 倍的 LLM 推理加速

Published on August 11, 2026 by Francesco Bonacci and Johnny Franks 发布于 2026 年 8 月 11 日，作者：Francesco Bonacci 和 Johnny Franks

If you’ve been following Cua from the start, you may remember that it began with a Show HN launch for Lume, our macOS virtualization stack. A macOS guest running through Apple’s Virtualization.framework uses a virtual GPU backed by the host’s Apple GPU. In our stock Tahoe VM, that device reported a conservative Metal capability profile. Applications use those answers to select kernels and rendering paths, which left llama.cpp running much slower GPU code. 如果你从一开始就关注 Cua，你可能还记得它始于我们 macOS 虚拟化栈 Lume 的 Show HN 发布。通过 Apple 的 Virtualization.framework 运行的 macOS 客户机使用由宿主机 Apple GPU 支持的虚拟 GPU。在我们标准的 Tahoe 虚拟机中，该设备报告了一个保守的 Metal 能力配置。应用程序会根据这些反馈来选择内核和渲染路径，这导致 llama.cpp 运行了速度较慢的 GPU 代码。

We built a small, process-scoped compatibility layer that changes selected capability answers for one guest process, allowing llama.cpp to select newer Metal kernels. This is the first result from our broader effort to connect Lume’s virtualization foundation to the local computer-use environments behind Cua Driver and the infrastructure behind Cua Cloud and Fleets. We’re releasing this work today as a research release under the same permissive license as Lume and Cua, so others can reproduce the results and help map which Apple Silicon chips, macOS releases, and Metal workloads benefit. 我们构建了一个小型、进程级的兼容层，用于更改特定客户机进程的选定能力反馈，从而允许 llama.cpp 选择更新的 Metal 内核。这是我们将 Lume 虚拟化基础与 Cua Driver 后端的本地计算机使用环境，以及 Cua Cloud 和 Fleets 基础设施相连接的更广泛努力的首个成果。我们今天以研究版本的形式发布这项工作，并采用与 Lume 和 Cua 相同的宽松许可，以便其他人可以复现这些结果，并帮助梳理哪些 Apple Silicon 芯片、macOS 版本和 Metal 工作负载能够从中受益。

On an M1 Ultra, TinyLlama 1.1B running through llama.cpp processed prompts 11.08× faster and generated tokens 16.36× faster than the same workload in the same stock VM. Prompt processing reached 98% of our bare-metal result. The source, build scripts, capability probe, and raw benchmark logs are included so you can inspect and reproduce the result. 在 M1 Ultra 上，通过 llama.cpp 运行的 TinyLlama 1.1B 处理提示词的速度比在同一标准虚拟机中运行相同工作负载快 11.08 倍，生成 Token 的速度快 16.36 倍。提示词处理速度达到了我们裸机测试结果的 98%。我们提供了源代码、构建脚本、能力探测工具和原始基准测试日志，以便你检查并复现该结果。

We repeated the experiment with Google’s Gemma 4 12B QAT Q4_0, a 6.98 GB model released this year. The same layer improved prompt processing 7.20× and token generation 14.54×. The unlocked VM reached 99.59% of bare-metal prompt speed and 94.82% of bare-metal generation speed. 我们使用今年发布的 Google Gemma 4 12B QAT Q4_0（一个 6.98 GB 的模型）重复了该实验。同样的兼容层使提示词处理速度提升了 7.20 倍，Token 生成速度提升了 14.54 倍。解锁后的虚拟机达到了裸机提示词处理速度的 99.59% 和生成速度的 94.82%。

We then tested Meta’s official Muse Glimmer 30B Q4_K-M GGUF in a 64 GiB guest. Through llama.cpp b10359, the unlocked VM processed a 512-token prompt 7.55× faster and generated 128 tokens 8.87× faster than the stock guest. This was a text-only llama.cpp test; it did not use Ollama, a multimodal projector, or a drafter. The same capability gap has surfaced in other Virtualization.framework frontends. Tart, another macOS virtualization CLI, has an open “No GPU passthrough in macOS guest?” issue covering graphics and LLM performance inside macOS guests. 随后，我们在 64 GiB 内存的客户机中测试了 Meta 官方的 Muse Glimmer 30B Q4_K-M GGUF 模型。通过 llama.cpp b10359，解锁后的虚拟机处理 512 个 Token 的提示词速度比标准客户机快 7.55 倍，生成 128 个 Token 的速度快 8.87 倍。这是一个纯文本的 llama.cpp 测试；未使用 Ollama、多模态投影仪或草稿模型。同样的性能差距也出现在其他 Virtualization.framework 前端中。另一个 macOS 虚拟化命令行工具 Tart 也有一个名为“macOS 客户机中没有 GPU 直通？”的开放议题，涵盖了 macOS 客户机内部的图形和 LLM 性能问题。

The cap inside a macOS VM

macOS 虚拟机内部的限制

Apple’s Virtualization.framework presents a macOS guest with a virtual graphics device. The guest submits Metal work through a purpose-built GPU driver, and Apple’s host stack executes it on the physical GPU. This arrangement is paravirtualization, where the host keeps control of the hardware and the guest uses a virtualization-aware device. This differs from other virtualization stacks built on QEMU and KVM, which can use a different architecture. On x86 Linux hosts, VFIO can assign a compatible physical PCI device or hardware function to a VM through an IOMMU, giving the guest direct access to that device. This is the model usually meant by GPU passthrough. Apple 的 Virtualization.framework 为 macOS 客户机提供了一个虚拟图形设备。客户机通过专用的 GPU 驱动程序提交 Metal 任务，Apple 的宿主机栈则在物理 GPU 上执行这些任务。这种安排属于半虚拟化（paravirtualization），即宿主机保持对硬件的控制，而客户机使用感知虚拟化的设备。这与基于 QEMU 和 KVM 构建的其他虚拟化栈不同，后者可以使用不同的架构。在 x86 Linux 宿主机上，VFIO 可以通过 IOMMU 将兼容的物理 PCI 设备或硬件功能分配给虚拟机，从而使客户机能够直接访问该设备。这通常就是所谓的“GPU 直通”模型。

In our stock Tahoe VM, the paravirtualized device reported roughly an Apple 5-era family, 32 KB of maximum threadgroup memory, and SIMD-group matrix support as unavailable. Modern Metal software uses those answers to select kernels, so llama.cpp took a slower path even though the device could execute newer kernels. Apple documents GPU capability through GPU families and feature tables and recommends querying the device at runtime. That makes the reported capability boundary consequential: applications are doing exactly what the platform tells them to do. 在我们标准的 Tahoe 虚拟机中，半虚拟化设备报告的性能大致相当于 Apple 5 系列，最大线程组内存为 32 KB，且 SIMD 组矩阵支持不可用。现代 Metal 软件会根据这些反馈来选择内核，因此尽管设备本身能够执行更新的内核，llama.cpp 却选择了较慢的路径。Apple 通过 GPU 系列和功能表记录 GPU 能力，并建议在运行时查询设备。这使得报告的能力边界变得至关重要：应用程序完全按照平台告知它们的内容行事。

The solution: a process-scoped Metal capability shim

解决方案：进程级的 Metal 能力垫片（Shim）

We built a small Metal capability shim (a compatibility layer inserted between an application and an API) that runs inside one guest process. It intercepts selected Metal capability queries and changes the answers returned to that process. Metal applications use those answers to select kernels, so returning the tested Apple-family and threadgroup-memory values lets llama.cpp choose its newer GPU paths. 我们构建了一个小型 Metal 能力垫片（插入应用程序和 API 之间的兼容层），它运行在单个客户机进程内。它会拦截选定的 Metal 能力查询，并更改返回给该进程的答案。Metal 应用程序使用这些答案来选择内核，因此返回经过测试的 Apple 系列和线程组内存值，可以让 llama.cpp 选择其更新的 GPU 路径。

For our tested profile, the shim: answers supportsFamily: through Apple family 9 (1009); and raises the reported maximum threadgroup memory from 32 KB to 64 KB. That was enough for the tested llama.cpp build to select newer SIMD-group reduction, SIMD-group matrix, and bfloat16 paths. 对于我们测试的配置，该垫片：将 supportsFamily 的反馈提升至 Apple family 9 (1009)；并将报告的最大线程组内存从 32 KB 提高到 64 KB。这足以让测试的 llama.cpp 构建版本选择更新的 SIMD 组归约（reduction）、SIMD 组矩阵和 bfloat16 路径。

(Table omitted for brevity, but the logic remains: the shim unlocks advanced features by reporting higher capabilities.) (表格此处略，但逻辑一致：垫片通过报告更高的能力解锁了高级功能。)

The tested profile changes two reported values: Apple-family answers and the threadgroup-memory limit. Common, Mac, Metal, and working-set-size values keep their stock settings during the benchmark. We removed the original research hook’s private feature-profile hook, clock and timing interposition, mesh substitution, ray-tracing override, argument-layout guard, and pipeline-compilation fallback. Its source is small enough to audit, and malformed or missing configuration keeps the process on its stock capability path. 测试配置更改了两个报告值：Apple 系列反馈和线程组内存限制。在基准测试期间，Common、Mac、Metal 和工作集大小（working-set-size）值保持其默认设置。我们移除了原始研究钩子中的私有功能配置钩子、时钟和计时干预、网格替换、光线追踪覆盖、参数布局保护以及管线编译回退。其源代码足够精简，易于审计，且如果配置格式错误或缺失，进程将保持在默认的能力路径上。

The workload stays on Apple’s Virtualization.framework graphics path and executes on the host’s Apple GPU. The capability changes are scoped to the injected guest process. Physical GPU assignment, raw PCI or VFIO passthrough, and kernel changes sit outside this mechanism. A reported family describes the paths covered by our tests; each additional Metal API requires separate validation. The shim unlocks Metal capabilities on Apple’s existing virtual GPU path. VM users often encounter the broader limitation under the name “GPU passthrough.” 工作负载保留在 Apple 的 Virtualization.framework 图形路径上，并在宿主机的 Apple GPU 上执行。能力更改仅限于注入的客户机进程。物理 GPU 分配、原始 PCI 或 VFIO 直通以及内核更改均不属于此机制。报告的系列描述了我们测试所涵盖的路径；每个额外的 Metal API 都需要单独验证。该垫片在 Apple 现有的虚拟 GPU 路径上解锁了 Metal 能力。虚拟机用户通常以“GPU 直通”的名义遇到更广泛的限制。

Fresh result from the minimal artifact

来自最小化制品的最新结果

We tested on one Apple M1 Ultra with a 48-core GPU and macOS 26.6.1. The guest was the current public Tahoe Cua image (macOS 26.5.2, 8 vCPU, and 16 GiB) running in Lume 0.5.1. All three runs used the official llama.cpp b10167 release and the same TinyLlama 1.1B Chat Q4_K_M model. The command was: llama-bench -m tinyllama-1.1b-chat-v1. 我们在配备 48 核 GPU 和 macOS 26.6.1 的 Apple M1 Ultra 上进行了测试。客户机是当前公开的 Tahoe Cua 镜像（macOS 26.5.2，8 vCPU，16 GiB），运行在 Lume 0.5.1 中。所有三次运行均使用官方 llama.cpp b10167 版本和相同的 TinyLlama 1.1B Chat Q4_K_M 模型。命令为：llama-bench -m tinyllama-1.1b-chat-v1.