Tokens Too Cheap to Meter

代币价格低至无需计量

Table of contents 目录

Is this really happening? Improvements that affect all AI, Improvements that affect hosted AI, Improvements that affect local AI, Improvements that affect specialized use cases, Putting it together, What happens next? Tokens become cheaper than tool calls, Supply-side Jevons Paradox, How are investors going to make their money back? Demand-side Jevons Paradox, Optionality, Summary.

这真的在发生吗？影响所有 AI 的改进、影响托管 AI 的改进、影响本地 AI 的改进、影响特定用例的改进、综合分析、接下来会发生什么？代币变得比工具调用更便宜、供给侧杰文斯悖论、投资者将如何回本？需求侧杰文斯悖论、可选性、总结。

The price of using machine learning intelligence is decreasing by several orders of magnitude a year and shows no signs of slowing. We are likely to see LLMs integrated into every part of computing as infrastructure, not just as a product, in the next year or two. We are likely to see LLMs running locally at current frontier-quality on commodity hardware in the next 3-6 years. Starting very soon, we are likely to see quality and access become the limiting factor to AI use, not sheer number of tokens.

使用机器学习智能的价格每年都在以几个数量级的速度下降，且没有放缓的迹象。在未来一两年内，我们很可能会看到大语言模型（LLM）作为基础设施而非仅仅作为产品，集成到计算的每一个环节中。在未来 3 到 6 年内，我们很可能会看到当前前沿质量的 LLM 在普通硬件上本地运行。很快，限制 AI 使用的因素将不再是代币的数量，而是质量与获取途径。

Is this really happening? 这真的在发生吗？

Extraordinary claims require extraordinary evidence, so I collected a whole bunch of evidence. AI can be either proprietary (such as GPT-6 Astra) or open weight (such as GLM-5.3-flash). Open weight models can be either hosted (e.g. by Z.ai) or local. Generally, models intended to be run locally will be much smaller, such as Muse Glimmer or Qwen3 Coder. Improvements in one don’t always affect improvements in the others.

非凡的声明需要非凡的证据，所以我收集了大量的证据。AI 可以是专有的（如 GPT-6 Astra），也可以是开放权重的（如 GLM-5.3-flash）。开放权重模型既可以是托管的（例如由 Z.ai 托管），也可以是本地运行的。通常，旨在本地运行的模型会小得多，例如 Muse Glimmer 或 Qwen3 Coder。某一领域的改进并不总是会影响其他领域。

Improvements that affect all AI 影响所有 AI 的改进

GPUs GPU

GPUs are getting exponentially more efficient with every generation. In the graph below (source), the X-axis is time and the Y-axis is power efficiency of the GPU itself. Larger Y-axis numbers mean more efficient. This is a logarithmic graph, which is to say that a straight line on the graph represents an exponential increase in efficiency. In this particular case, the logarithm is 1.3, which means efficiency doubles about once every two years. This is an increase in efficiency that we haven’t seen since Moore’s Law in the 1960s.

GPU 的效率随着每一代产品的更迭呈指数级增长。在下方的图表中（来源），X 轴代表时间，Y 轴代表 GPU 本身的能效。Y 轴数值越大，效率越高。这是一个对数图，这意味着图中的直线代表效率的指数级增长。在本例中，对数基数为 1.3，这意味着效率大约每两年翻一番。这种效率提升是我们自 20 世纪 60 年代摩尔定律以来从未见过的。

Models 模型

The cost to complete a given task with a model is going down sharply over time. Models are usually priced per-token. A “token” is a fragment of a word; it takes about 1.5 tokens to represent a word. For every token a model reads, and for every token it outputs, the “model provider” (e.g. Anthropic or OpenAI) charges you some fixed amount of money. The cost per token of models is not consistently going down, at least not for the smartest (“frontier”) models. But the cost per task is. Smaller models may cost less per token, but use more tokens overall than a larger model for the same task, because they have to think more or correct their first drafts. This section is about the cost to complete the task from beginning to end.

使用模型完成特定任务的成本正在随时间急剧下降。模型通常按代币计价。“代币”是单词的片段；大约需要 1.5 个代币来表示一个单词。模型每读取一个代币，每输出一个代币，“模型提供商”（如 Anthropic 或 OpenAI）都会向你收取固定的费用。模型的单位代币成本并非持续下降，至少对于最智能的（“前沿”）模型而言并非如此。但完成任务的成本却在下降。较小的模型可能单位代币成本较低，但完成同一任务时使用的代币总量可能比大型模型更多，因为它们需要更多的思考或修正初稿。本节讨论的是从头到尾完成任务的成本。

The chart below (source) shows the “pareto frontier” of cost/task at present. A pareto frontier shows the best tradeoff you can get, not just the best in a single category. Here, our tradeoffs are: Y-axis: the “quality” of the model (as measured by a suite of benchmarks); X-axis: the cost to complete those benchmarks. Cost is on a logarithmic scale. Larger Y-axis and smaller X-axis numbers are better. This is showing us a wide range of models on the pareto frontier as of 2026. Towards the top-right we have Claude Fable-5.1 (expensive and intelligent); towards the middle-left we have GPT-5.6 Luna (cheap and less intelligent). Models below the dotted line are basically not worth considering.

下方的图表（来源）展示了当前任务成本的“帕累托前沿”。帕累托前沿展示了你能获得的最佳权衡，而不仅仅是单一类别的最优值。在这里，我们的权衡指标是：Y 轴为模型的“质量”（通过一系列基准测试衡量）；X 轴为完成这些基准测试的成本。成本采用对数刻度。Y 轴数值越大、X 轴数值越小越好。这向我们展示了截至 2026 年帕累托前沿上的各种模型。在右上角，我们有 Claude Fable-5.1（昂贵且智能）；在左中位置，我们有 GPT-5.6 Luna（便宜且智能程度较低）。虚线以下的模型基本不值得考虑。

Now, look at this chart showing the frontier at the start, middle, and end of 2025: The chart shows models are getting smarter and cheaper on a per-task basis over 2025. If you draw a straight horizontal line at basically any task on the Y-axis, the cost to do it at the end of 2025 was cheaper than at the start; and if you draw a straight vertical line at basically any point on the X-axis, models can do more for the same cost. Now, compare that 2025 chart to the 2026 chart. The Y-axis (intelligence) is about the same, with less of a fall-off towards the cheap end. The X-axis (cost) has gotten two orders of magnitude cheaper.

现在，看看这张展示 2025 年初、年中和年末前沿水平的图表：该图表显示，在 2025 年期间，模型在单任务基础上的智能程度在提高，成本在下降。如果你在 Y 轴的任何任务上画一条水平线，2025 年末完成该任务的成本都比年初更低；如果你在 X 轴的任何点画一条垂直线，模型在相同成本下能做的事情更多。现在，将 2025 年的图表与 2026 年的图表进行比较。Y 轴（智能）大致相同，但在廉价端性能下降较少。X 轴（成本）则降低了两个数量级。

Inference Engines 推理引擎

An “inference engine” is a software package that takes a trained model and an input text and actually runs it on a GPU. Inference engines are currently immature and improving rapidly. Currently we’re seeing 10%-50% improvements year-over-year, depending on which engine you look at. There are two benchmarks that are often compared for inference engines: “offline” (run a bunch of tokens through in one big batch) and “serving” (you have people sending your server inputs at unpredictable times, and you want to send a response back as quickly as possible). Serving is getting efficient much more rapidly than offline inference. All numbers below are for serving workloads, not offline.

“推理引擎”是一个软件包，它接收训练好的模型和输入文本，并在 GPU 上实际运行它。推理引擎目前尚不成熟，但正在迅速改进。目前，根据引擎的不同，我们每年能看到 10%-50% 的性能提升。推理引擎通常有两个基准测试：“离线”（一次性批量处理大量代币）和“服务”（用户在不可预测的时间向服务器发送输入，而你希望尽快返回响应）。服务模式的效率提升速度远快于离线推理。以下所有数据均针对服务工作负载，而非离线工作负载。

vLLM vLLM

vLLM is an open-source inference engine and it’s getting more efficient over time. In the graph below (source), the Y-axis is Joules/token, the X-axis is batch size (roughly: “how many inputs are processed in parallel?”), and the blue/red lines are different software versions. Smaller Y-axis numbers mean more efficient. vLLM 0.11.1 was released in December 2025, a bit more than a year after vLLM 0.5.4 in September 2024. In other words, this is about a 40% increase in efficiency in 15 months. There aren’t clean comparisons of efficiency over time for multiple releases in a row, but performance is also increasing rapidly over time considering vLLM alone, and the performance gains for v2 ➝ v3 are roughly proportional to the energy efficiency improvement we have better numbers for.

vLLM 是一个开源推理引擎，其效率正随时间推移而提高。在下方的图表中（来源），Y 轴是焦耳/代币，X 轴是批处理大小（大致为：“有多少输入被并行处理？”），蓝色/红色线条代表不同的软件版本。Y 轴数值越小，效率越高。vLLM 0.11.1 发布于 2025 年 12 月，距离 2024 年 9 月发布的 vLLM 0.5.4 仅一年多一点。换句话说，这在 15 个月内实现了约 40% 的效率提升。虽然没有连续多个版本随时间推移的效率对比，但仅考虑 vLLM 本身，性能也在迅速提升，且 v2 到 v3 的性能增益与我们掌握的能效提升数据大致成正比。

NVIDIA 英伟达

This isn’t isolated to a single software package. NVIDIA is showing up to 50% efficiency improvements on their MLPerf stack from 2.0 to 2.1:

这并非单一软件包的孤立现象。英伟达在 MLPerf 堆栈从 2.0 升级到 2.1 的过程中，效率提升高达 50%：

Intel 英特尔

This isn’t isolated to old benchmarks. Intel recently showed a 2.4x throughput increase solely by improving MLPerf between 6.0 and 6.1. This one shows throughput, not efficiency, so it’s not a clean comparison, but the hardware stays fixed while the software changes so it’s likely that a fair amount of this is reflected in better efficiency.

这并非仅限于旧的基准测试。英特尔最近展示了仅通过改进 MLPerf 6.0 到 6.1 之间的性能，吞吐量就提升了 2.4 倍。这显示的是吞吐量而非效率，因此不是一个精确的对比，但由于硬件保持不变而软件发生了变化，很可能其中很大一部分反映在效率的提升上。