The Energy and Water Cost of Inference

推理的能源与水资源成本

Estimates of the energy in one model query differ by orders of magnitude across credible sources. Most of that spread is not disagreement about physics. It is disagreement about where the system boundary is drawn, and a figure quoted without its boundary is not a figure. 关于单次模型查询所需能量的估算，不同权威来源之间存在数量级的差异。这种差异大多并非源于物理学上的分歧，而是源于对“系统边界”定义的不同。一个没有明确边界定义的数值，实际上并无意义。

Why the published figures disagree: Before comparing two numbers, establish which of these each one includes. Any of them can change the answer by more than the model choice does: 为何已发布的数值会存在分歧？在比较两个数字之前，必须先确定它们各自包含了哪些部分。以下任何一项对结果的影响，都可能超过模型选择本身：

• Which model, and how much output. A short answer from a small model and a long answer from a large reasoning model differ by several orders of magnitude on their own. A single “per query” figure averages over a distribution nobody specifies. 模型类型与输出长度。 小型模型的简短回答与大型推理模型的长篇回答，其能耗差异可达数个数量级。单一的“单次查询”数值是对某种未定义的分布取平均值的结果。
• Batch size and utilisation. The dominant engineering term. Serving many requests concurrently amortises the weight read across all of them; the same hardware at low occupancy spends nearly the same power for a fraction of the tokens. 批处理大小与利用率。 这是工程层面的决定性因素。并发处理多个请求可以将权重读取的成本分摊到所有请求中；而在低负载下，同样的硬件在处理极少数 Token 时，其功耗几乎与高负载时相同。
• Facility overhead. Cooling, power conversion and distribution, captured as power usage effectiveness (PUE). It multiplies everything, and whether a figure includes it is frequently unstated. 设施开销。 包括冷却、电力转换和分配，通常以电源使用效率（PUE）衡量。它会作用于所有能耗项，但数据是否包含此项往往未被说明。
• Training amortisation. Some analyses divide training energy across expected lifetime queries. Defensible, and it produces a different quantity from marginal serving energy. They are not comparable. 训练成本分摊。 一些分析将训练能耗分摊到预期的生命周期查询次数中。这种做法有其合理性，但它得出的数值与“边际推理能耗”是完全不同的概念，两者不可直接比较。
• Embodied energy. Manufacturing the accelerators, the building and the power infrastructure. Usually excluded, occasionally included, rarely flagged. 隐含能源（Embodied energy）。 指制造加速器、建筑和电力基础设施所需的能源。通常被排除在外，偶尔包含，但极少被明确标注。
• Idle and provisioned capacity. Capacity is held for peak. Charging queries only for the seconds they compute understates the system; charging them for provisioned capacity overstates the marginal query. Both are used. 闲置与预留容量。 系统容量是为峰值需求而预留的。仅计算查询计算期间的耗电会低估系统成本；而将预留容量的成本分摊到单次查询中又会高估边际成本。这两种方法都有人在用。

Building the estimate yourself

如何自行估算

The marginal serving calculation is not complicated, and doing it once makes every published figure legible. 边际推理能耗的计算并不复杂，亲自计算一次后，你就能看懂所有已发布的报告。

Serving side, per accelerator: 推理侧，单加速器计算公式： E_per_token = (P_device · n_devices · PUE) / R_tokens_per_second

• P_device: average power draw under load, from the spec sheet (below the rated maximum in practice). P_device：负载下的平均功耗，参考规格说明书（实际使用中通常低于额定最大值）。
• n_devices: accelerators the model is sharded across. n_devices：模型分片所跨越的加速器数量。
• PUE: facility overhead multiplier. PUE：设施开销乘数。
• R_tokens_per_second: TOTAL tokens per second across the whole batch, not per request ← this is the term that dominates. R_tokens_per_second：整个批次的每秒总 Token 数，而非单个请求的 Token 数——这是决定性因素。

Per request: 单次请求能耗： E_request ≈ E_prefill(prompt_tokens) + E_per_token · output_tokens 对于推理模型，output_tokens 包含了“思考过程”产生的 Token，这部分往往占据了绝大多数。

The structure of the expression is the lesson. Energy per token is accelerator power divided by aggregate throughput, so anything that raises throughput at fixed power lowers energy per token proportionally. Batching is that lever, and it is why a busy, well-loaded deployment is far more energy-efficient per token than a lightly used one — the opposite of the intuition that heavy use is the problem. 这个公式揭示了核心逻辑：单位 Token 能耗等于加速器功耗除以总吞吐量。因此，在功耗固定的前提下，任何能提高吞吐量的手段都能按比例降低单位 Token 能耗。批处理就是关键杠杆，这就是为什么高负载、充分利用的部署在单位 Token 能耗上远比低负载部署更高效——这与“高负载会导致能耗问题”的直觉恰恰相反。

Which terms actually move it

哪些因素真正影响能耗

• Output length: Linear, and the term a user controls. Asking for a concise answer is an energy decision as much as a latency one. Reasoning models can spend far more tokens than they return. 输出长度： 呈线性关系，且由用户控制。要求简洁回答不仅是出于延迟考虑，也是一种节能决策。推理模型在思考过程中消耗的 Token 往往远多于最终输出的 Token。
• Batch size / utilisation: Large and non-linear at the low end. Most of the difference between a well-run and a badly-run deployment is here, not in the model. 批处理大小/利用率： 影响巨大，且在低负载端呈非线性。运行良好的部署与运行糟糕的部署之间，大部分差异源于此，而非模型本身。
• Active parameters: Drives arithmetic and memory traffic per token. Sparse models decouple this from total size, which is why headline parameter counts predict energy poorly. 活跃参数： 决定了每个 Token 的算术运算量和内存流量。稀疏模型将此与总参数量解耦，这就是为什么仅凭参数总量很难预测能耗的原因。
• Precision and quantisation: Lower precision reduces both memory traffic and arithmetic energy, usually the cheapest real reduction available. 精度与量化： 降低精度可同时减少内存流量和算术能耗，通常是目前最廉价的节能手段。
• Prompt caching: Removes repeated prefill for a shared prefix. Where prompts are long and repetitive this is a large share of total work. 提示词缓存（Prompt caching）： 避免了对共享前缀的重复预填充。在提示词冗长且重复的情况下，这能节省大量工作量。
• Routing to model size: Sending easy requests to a smaller model is the largest available reduction for most applications, and the one that requires actual engineering rather than a setting. 模型路由： 将简单请求分发给较小的模型，是大多数应用中能实现的最大节能手段，这需要真正的工程设计，而不仅仅是调整设置。

Water is two different numbers

水资源消耗的两个维度

Water figures are the most confused part of this discussion because two unrelated quantities are reported under one word. 水资源消耗数据是讨论中最容易混淆的部分，因为“用水量”这个词涵盖了两个互不相关的概念。

On-site consumption is water evaporated by cooling. It depends on the cooling design (evaporative systems consume water and save electricity, closed-loop and air-cooled systems do the reverse) and on climate and season. It is a genuine local resource question in a water-stressed region, and it is a design choice rather than an inherent property of computing. 现场消耗（On-site consumption） 是指冷却过程中蒸发的水量。它取决于冷却设计（蒸发式系统耗水省电，闭环或风冷系统则相反）以及气候和季节。在缺水地区，这是一个真实的本地资源问题，且属于设计选择，而非计算本身的固有属性。

Off-site water is attributed to generating the electricity, mostly thermoelectric cooling at power stations. It usually dwarfs the on-site figure and it depends on the generation mix rather than on the data centre at all. Analyses reporting a striking water number often include it. Neither is wrong. Reporting one without saying which is. 非现场消耗（Off-site water） 是指发电过程中消耗的水量，主要用于发电厂的热电冷却。这部分通常远超现场用水量，且完全取决于能源结构，与数据中心本身无关。那些报告惊人用水量的分析往往包含了这一项。两者都没有错，但如果不加说明地混用，就是错误的。

Per-query is the wrong unit for policy

“单次查询”并非政策制定的合适单位

A per-query figure is useful for personal reasoning and nearly useless for policy. What a grid experiences is aggregate load with a time and location profile — how much power, drawn where. “单次查询”的能耗数据对个人决策有参考价值，但对政策制定几乎毫无用处。电网所面对的是具有时间与空间特征的聚合负载——即在何时、何地消耗了多少电力。