Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac

Show HN：开源引擎，可在任何 M 系列 Mac 上以 2 GB 内存运行 Gemma 4 26B

TurboFieldfare Gemma 4 26B-A4B inference in about 2 GB of RAM. A custom Swift + Metal runtime for any Apple Silicon Mac, even the 8 GB ones. Quick start · Local server · Benchmarks · Contribute results · How it works · Experiments · References.

TurboFieldfare 可在约 2 GB 内存中进行 Gemma 4 26B-A4B 推理。这是一个为所有 Apple Silicon Mac（甚至是 8 GB 内存版本）定制的 Swift + Metal 运行时。快速入门 · 本地服务器 · 基准测试 · 贡献结果 · 工作原理 · 实验 · 参考资料。

Memory got expensive. So I gave a 26-billion-parameter model a ~2 GB budget. TurboFieldfare runs the instruction-tuned Gemma 4 26B-A4B without loading the entire 14.3 GB model into memory. It keeps the shared 1.35 GB core and FP16 KV cache in memory, then streams only the experts needed for each token from SSD. This is what lets the model run on Macs with 8 GB of RAM.

内存变得昂贵了，所以我给一个 260 亿参数的模型设定了约 2 GB 的预算。TurboFieldfare 在运行指令微调版 Gemma 4 26B-A4B 时，无需将整个 14.3 GB 的模型加载到内存中。它仅将 1.35 GB 的共享核心和 FP16 KV 缓存保留在内存中，然后从 SSD 流式传输每个 token 所需的专家模型。这就是该模型能在 8 GB 内存的 Mac 上运行的原因。

The runtime, streaming installer, CLI, and native Mac app are written in Swift and Metal. TurboFieldfare is model-specific rather than a wrapper around MLX or llama.cpp. The curated experiment record summarizes 103 measured results across kernels, caching, I/O, prefill, and decode.

该运行时、流式安装程序、CLI 和原生 Mac 应用均使用 Swift 和 Metal 编写。TurboFieldfare 是针对特定模型的，而不是 MLX 或 llama.cpp 的封装。精心整理的实验记录总结了涵盖内核、缓存、I/O、预填充（prefill）和解码（decode）的 103 项测量结果。

Try it / 尝试使用

git clone https://github.com/drumih/turbo-fieldfare.git
cd turbo-fieldfare
swift build -c release
.build/release/TurboFieldfareMac

On the first run, Swift Package Manager downloads and builds the Swift packages required by the tokenizer. The complete release build includes the foreground Mac app and its sibling decode-service executable. When the app opens, choose Download and let TurboFieldfare fetch and repack the pinned model (about 15 GB). Once it is ready, choose Load Model, type your prompt, and press Generate.

首次运行时，Swift Package Manager 会下载并构建分词器所需的 Swift 包。完整的发布版本包含前台 Mac 应用及其配套的 decode-service 可执行文件。当应用打开时，选择“Download”，让 TurboFieldfare 获取并重新打包指定的模型（约 15 GB）。准备就绪后，选择“Load Model”，输入提示词，然后点击“Generate”。

At a glance / 概览

Metric	Value
Model	Gemma 4 26B-A4B IT, 26B total parameters, about 3.88B active per token
Weights	MLX affine 4-bit, group 64; 8-bit router; 4-bit shared and routed experts
Memory	~2 GB of weights and 4K KV cache
Storage	About 14.3 GB for the installed text-only model
Hardware	Apple Silicon Mac; 8 GB of RAM
Platform	macOS 26, Metal 4, Swift 6.2

指标	数值
模型	Gemma 4 26B-A4B IT，总参数 26B，每个 token 激活约 3.88B
权重	MLX 仿射 4-bit，组 64；8-bit 路由；4-bit 共享及路由专家
内存	约 2 GB 权重及 4K KV 缓存
存储	约 14.3 GB（仅文本模型安装空间）
硬件	Apple Silicon Mac；8 GB 内存
平台	macOS 26, Metal 4, Swift 6.2

M2 measured decode 5.1-6.3 tok/s on an 8 GB M2 MacBook Air. M5 measured decode 31-35 tok/s on a 24 GB M5 Pro. The measured result is a reference point, not a performance ceiling. Prompt length, generated length, page-cache state, and hardware all affect throughput. To help measure another Apple Silicon Mac, follow the community benchmark guide.

在 8 GB M2 MacBook Air 上测得解码速度为 5.1-6.3 tok/s。在 24 GB M5 Pro 上测得解码速度为 31-35 tok/s。测量结果仅供参考，并非性能上限。提示词长度、生成长度、页面缓存状态和硬件都会影响吞吐量。如需协助测量其他 Apple Silicon Mac，请参考社区基准测试指南。

Using TurboFieldfare / 使用 TurboFieldfare

TurboFieldfare provides a native Mac app, a command-line interface, and an experimental loopback OpenAI-compatible server. They use the same .gturbo model directory, but only one model-owning product should run at a time.

TurboFieldfare 提供原生 Mac 应用、命令行界面以及一个实验性的回环 OpenAI 兼容服务器。它们使用相同的 .gturbo 模型目录，但同一时间只能运行一个占用模型的程序。

Requirements / 要求

An Apple Silicon Mac; the validated target is an 8 GB M2 MacBook Air
macOS 26 with Metal 4
Xcode 26 and Swift 6.2 or newer
Enough free storage for the ~14.3 GB model installation
An internet connection for the first model install
Apple Silicon Mac；验证目标为 8 GB M2 MacBook Air
macOS 26 及 Metal 4
Xcode 26 和 Swift 6.2 或更高版本
足够的剩余存储空间以安装约 14.3 GB 的模型
首次安装模型时需要互联网连接

The package is arm64-only. Older macOS and Metal versions are not supported.

该包仅支持 arm64 架构。不支持旧版本的 macOS 和 Metal。

Prompting the model / 模型提示

The Mac app treats what you type as an instruction and handles Gemma’s chat formatting automatically. Just describe the task and include any context the model needs. Generation defaults to temperature 0.2, Top-K 64, and Top-P 0.95. Set temperature to 0 for deterministic greedy output.

Mac 应用会将你输入的内容视为指令，并自动处理 Gemma 的聊天格式。只需描述任务并包含模型所需的任何上下文即可。生成默认参数为：温度 0.2，Top-K 64，Top-P 0.95。将温度设置为 0 可获得确定性的贪婪输出。

Mac app / Mac 应用

Clone the repository, then run the app from its root: 克隆仓库，然后从根目录运行应用：

swift build -c release
.build/release/TurboFieldfareMac

Build the complete package so the app and its sibling decode service are both available. When launched from this checkout, the app stores the model in scratch/gemma4.gturbo.

构建完整包，以便应用及其配套的解码服务均可用。从该检出目录启动时，应用会将模型存储在 scratch/gemma4.gturbo 中。

Install the model / 安装模型

On first launch, the app checks the available storage and shows the download and installed sizes. Choose Download to begin. The installer never materializes the full source checkpoint. It streams the required byte ranges from the pinned Hugging Face revision and repacks them directly into the .gturbo layout as they arrive. This avoids a second full checkpoint on disk and keeps scratch memory bounded.

首次启动时，应用会检查可用存储空间并显示下载及安装所需的大小。选择“Download”开始。安装程序不会完整下载源检查点，而是从指定的 Hugging Face 版本流式传输所需的字节范围，并在接收时直接将其重新打包为 .gturbo 布局。这避免了在磁盘上产生第二个完整检查点，并保持了暂存内存的受控。

The first installation transfers about 15 GB through bounded Hugging Face range requests. Network speed and Hugging Face response times vary, so it can take a while. The completed .gturbo installation occupies about 14.3 GB and is accepted only after its manifest and file hashes have been validated. Installation does not load the model into memory.

首次安装通过受限的 Hugging Face 范围请求传输约 15 GB 数据。网络速度和 Hugging Face 响应时间各不相同，因此可能需要一些时间。完成后的 .gturbo 安装占用约 14.3 GB，且仅在清单和文件哈希验证通过后才会被接受。安装过程不会将模型加载到内存中。