How Much Does a Local LLM Actually Cost to Run? I Measured Every Watt on Apple Silicon
How Much Does a Local LLM Actually Cost to Run? I Measured Every Watt on Apple Silicon
本地运行大语言模型到底要花多少钱？我测量了 Apple Silicon 上的每一瓦功耗
Large Language Models: Five models, sustained generation, real wall-socket energy at $0.31/kWh — and the surprise the RTX-3090 numbers predicted, only bigger. 大语言模型：五种模型、持续生成、按每千瓦时 0.31 美元的实际墙上插座电费计算——结果印证了 RTX-3090 数据所预示的惊喜，而且差距更大。
The folk wisdom about running LLMs on your own hardware is that it’s free. You bought the machine, the electricity is a rounding error, and every token after that is gravy. A recent Towards Data Science piece, How Much Does It Actually Cost to Run a Local LLM?, put a number on that gravy by sampling the power draw of an RTX 3090 and pricing it against a real electricity tariff. Its finding stuck with me: the most expensive model to run wasn’t the biggest one. Cost per token tracked throughput, not parameter count. 关于在自己的硬件上运行大语言模型，民间普遍认为这是免费的。你已经买了机器，电费只是个零头，之后生成的每一个 Token 都是纯赚的。最近《Towards Data Science》的一篇文章《运行本地大语言模型到底要花多少钱？》通过采样 RTX 3090 的功耗并结合实际电价，为这种“免费”算了一笔账。它的发现让我印象深刻：运行成本最高的模型并不是参数量最大的那个。每个 Token 的成本取决于吞吐量，而不是参数规模。
I don’t have a 3090. I run everything on an M3 Ultra Mac Studio with 96 GB of unified memory—no discrete GPU, no VRAM, just one pool of memory the CPU and GPU share. That’s a different enough machine that I wanted to know whether the same surprise holds, and by how much. So I measured it. Five models, sustained generation, real wall-socket energy, priced at my actual utility rate. The short version: the surprise doesn’t just hold on Apple Silicon, it gets bigger. My 120-billion-parameter model is about five times cheaper per token than a model a quarter its size. Here’s how I got the numbers, and why they land where they do. 我没有 3090。我所有的运行环境都是一台配备 96GB 统一内存的 M3 Ultra Mac Studio——没有独立显卡，没有显存，只有 CPU 和 GPU 共享的内存池。这台机器的架构差异很大，我想知道同样的“惊喜”是否依然存在，以及差距有多大。于是我进行了测量。五种模型、持续生成、实际墙上插座电耗，并按我的实际电价计费。简而言之：这种惊喜不仅在 Apple Silicon 上成立，而且差距被进一步拉大了。我的 1200 亿参数模型每个 Token 的成本大约是其四分之一大小模型的五分之一。以下是我得出这些数据的方法，以及它们为何呈现出这样的结果。
How the measurement works
测量方法
The instrument is a small tool I built called TokenWatt. It’s an OpenAI-compatible proxy built just for Apple Silicon: you point it at whatever local inference server you already run, and it forwards every request byte-for-byte while bracketing it with an energy measurement. It reads the whole-SoC power rails through Apple’s IOReport interface—the same counters Activity Monitor’s energy tab is built on—which means no sudo, no external instrumentation, just the numbers the chip reports about itself. 我构建了一个名为 TokenWatt 的小工具来进行测量。这是一个专为 Apple Silicon 开发的兼容 OpenAI 协议的代理：你只需将其指向你已经在运行的任何本地推理服务器，它就会在转发每个请求的同时，通过测量能量消耗来对其进行标记。它通过 Apple 的 IOReport 接口读取整个 SoC 的电源轨数据——这与“活动监视器”中能量选项卡所使用的计数器相同——这意味着无需 sudo 权限，无需外部仪器，只需读取芯片自身报告的数据即可。
It subtracts a rolling idle baseline so you’re measuring the marginal cost of the request, not the machine sitting there breathing, and it prices the result at your electricity rate. Mine is a flat $0.31/kWh. The honesty problem with any measurement like this is that the on-die计数器 (on-die counters) can drift, and there’s no way for a reader to know whether to trust them. So the one thing I insisted on—and the reason the tool exists at all—is calibration against ground truth. 它会减去一个滚动空闲基准值，这样你测量的是请求的边际成本，而不是机器待机时的功耗，并按你的电价计算结果。我的电价是每千瓦时 0.31 美元。此类测量面临的诚信问题在于，片上计数器可能会出现偏差，读者无法判断是否可信。因此，我坚持做的一件事——也是这个工具存在的意义——就是针对真实基准进行校准。
I ran every measurement in this article with the Mac plugged into a Shelly Plug US Gen4 that meters actual energy at the wall, and TokenWatt’s built-in calibration fit the SoC counters to it—no hand-tuning, and the same procedure anyone with a metering plug can rerun. Every number below carries a real error band from that fit; they land between ±2.6% and ±4.5%. When you see “$0.109 per million tokens,” that’s a wall-verified figure with a stated uncertainty, not a hopeful reading off an internal register. 我在本文中的每一次测量都将 Mac 连接到 Shelly Plug US Gen4 插座上，以测量墙上的实际能耗。TokenWatt 的内置校准功能将 SoC 计数器与该数据进行了拟合——无需手动调整，任何拥有计量插座的人都可以重复此过程。下面每个数字都带有该拟合产生的真实误差范围；误差在 ±2.6% 到 ±4.5% 之间。当你看到“每百万 Token 0.109 美元”时，这是一个经过墙上插座验证且带有明确不确定性的数字，而不是从内部寄存器中读取的乐观数值。
For the headline numbers I ran each model in a sustained generation loop—the machine doing nothing but decoding tokens as fast as it can—at three durations (120, 360, and 720 seconds), three passes each, with a 15-minute idle baseline between runs. That’s the cleanest, most reproducible signal you can get: a saturated GPU, no gaps, no cold starts. It’s a best case, and I’ll come back to what happens in messy real traffic later. 为了获得核心数据，我让每个模型在持续生成循环中运行——机器除了尽可能快地解码 Token 外什么都不做——分别在三个时长（120、360 和 720 秒）下运行，每个时长进行三次，并在运行之间留出 15 分钟的空闲基准时间。这是你能得到的最干净、最可重复的信号：GPU 满载，没有间隙，没有冷启动。这是最佳情况，稍后我会再讨论在混乱的实际流量中会发生什么。
The five models
五种模型
Everything here fits in 96 GB, which turns out to matter (more on that near the end). The models span from a 4-billion-parameter dense model up to a 120-billion-parameter mixture-of-experts, across different quantizations. “Footprint” is what the weights occupy in unified memory, and “tok/s” is sustained decode throughput. Cost is per million output (generated) tokens, at $0.31/kWh. 这里的所有模型都能装入 96GB 内存中，事实证明这很重要（文末会详细说明）。这些模型涵盖了从 40 亿参数的稠密模型到 1200 亿参数的混合专家模型（MoE），并采用了不同的量化方式。“占用空间”是指权重在统一内存中占用的空间，“tok/s”是持续解码吞吐量。成本按每百万输出（生成）Token 计算，电价为每千瓦时 0.31 美元。
| Model | Params | Type | Footprint | tok/s | $/1M out |
|---|---|---|---|---|---|
| Qwen3.5-4B | 4B | dense | 2.9 GB | 133.8 | $0.063 |
| Qwen3.6-35B-A3B | 35B | MoE (3B) | 35 GB | 76.0 | $0.087 |
| Qwen3-Coder-Next | ~80B | MoE (3B) | 60 GB | 65.0 | $0.103 |
| gpt-oss-120b | 120B | MoE (5B) | 59 GB | 74.0 | $0.109 |
| Qwen3.6-27B | 27B | dense | 28 GB | 21.5 | $0.554 |
I sorted that table by cost on purpose, because sorting it by size would scramble it. The cheapest model is the smallest, sure. But the most expensive model, by a factor of five to nine, is the 27-billion-parameter dense one—smaller than three of the four models it’s more expensive than. The 120B model undercuts it five to one. Parameter count simply doesn’t predict what a model costs to run here: the most expensive one is a mid-size dense model, and every model larger than it is cheaper. 我特意按成本对表格进行了排序，因为如果按大小排序，顺序会变得混乱。最便宜的模型确实是最小的，没错。但最昂贵的模型（比其他模型贵 5 到 9 倍）却是 270 亿参数的稠密模型——它比它所超越的四个模型中的三个还要小。1200 亿参数的模型成本仅为其五分之一。在这里，参数数量根本无法预测模型的运行成本：最昂贵的模型是一个中等规模的稠密模型，而所有比它大的模型反而更便宜。
Why the big model wins
为什么大模型反而胜出
The mechanism is the same one the RTX-3090 piece identified, and it’s worth stating plainly: the electricity cost of a token is watts divided by throughput. Two things set it—how much power the machine draws while generating, and how many tokens it produces per second—and the 27B dense model loses on both. Look at its two numbers. It draws 138 watts, the most of any model here, and it generates only 21.5 tokens per second, the fewest. 其机制与 RTX-3090 那篇文章中指出的相同，值得明确说明：一个 Token 的电费等于功率除以吞吐量。它由两件事决定——机器在生成时消耗多少功率，以及每秒产生多少个 Token——而 270 亿参数的稠密模型在这两方面都输了。看看它的两个数据：它消耗 138 瓦功率（是这里所有模型中最高的），每秒仅生成 21.5 个 Token（是这里最少的）。
The reason is the thing that governs single-stream decode on this hardware: to produce one token, the machine has to read every active weight out of memory, and generation speed is set by how many bytes that is divided by the chip’s memory bandwidth. A dense 8-bit model activates all 27 billion parameters for every token—it streams its entire ~28 GB of weights per token. That’s the most data movement of anything in the table, which is why it’s both the slowest and the most power-hungry: high bytes-per-token, low speed, the worst of both worlds. 原因在于该硬件上控制单流解码的机制：为了生成一个 Token，机器必须从内存中读取每一个活跃权重，而生成速度取决于权重字节数除以芯片的内存带宽。一个 8-bit 的稠密模型在生成每个 Token 时都会激活全部 270 亿个参数——它每个 Token 都要流式传输其全部约 28GB 的权重。这是表中所有模型中数据移动量最大的，这就是它既最慢又最耗电的原因：高字节/Token，低速度，这是最糟糕的情况。
The mixture-of-experts models break that link between size and bytes-per-token. gpt-oss-120b has 120 billion parameters sitting in memory, but for any given token a router activates only about 5 billion of them—so it reads a small fraction of its 59 GB per token, even though the whole thing is resident. Less data moved per token means more speed (74 tok/s) and less power (94 W). 混合专家模型（MoE）打破了规模与每 Token 字节数之间的联系。gpt-oss-120b 在内存中拥有 1200 亿个参数，但对于任何给定的 Token，路由器只激活其中约 50 亿个——因此，尽管整个模型都驻留在内存中，但它每个 Token 只读取其 59GB 总量中的一小部分。每个 Token 移动的数据更少，意味着速度更快（74 tok/s）且功耗更低（94 W）。