The 3× Token Bill We Didn’t See Coming

我们未曾预料到的 3 倍 Token 账单

Agentic AI: The 3× Token Bill We Didn’t See Coming 智能体 AI：我们未曾预料到的 3 倍 Token 账单

How a seemingly harmless move to a multi-agent architecture quietly tripled our LLM costs and what actually fixed it. 从单体架构转向多智能体架构这一看似无害的举动，是如何悄无声息地让我们的 LLM（大语言模型）成本翻了三倍，以及最终是什么解决了这个问题。

A few weeks ago I was going through our agentic application dashboard and noticed a spike in last week’s LLM usage, with traffic sitting at exactly the same level it had been for weeks. Initially we assumed it was a logging bug but it caught a real issue. It lined up almost exactly with an architectural change we’d shipped a few weeks earlier: we’d moved a chunk of our pipeline from a single-agent setup to a multi-agent one. 几周前，我在查看我们的智能体应用仪表盘时，注意到上周的 LLM 使用量出现了激增，而流量水平与过去几周完全持平。起初我们以为是日志记录出现了 Bug，但它实际上反映了一个真实存在的问题。这与我们几周前发布的一项架构变更几乎完全吻合：我们将流水线的一部分从单智能体设置迁移到了多智能体设置。

Different agents handling different parts of a task, coordinated through LangGraph, with a supervisor node deciding what happens next. On paper this was a clean win accomplishing better task decomposition, each agent doing one thing well instead of one model trying to do everything inside a single sprawling prompt. You’d expect usage to go up somewhat, more calls per task is the cost of that architecture, and nobody was pretending otherwise. But to our surprise it had roughly tripled for tasks that functionally hadn’t changed at all. 不同的智能体处理任务的不同部分，通过 LangGraph 进行协调，并由一个监督节点决定下一步的操作。从理论上讲，这是一种完美的方案，实现了更好的任务分解——每个智能体各司其职，而不是让一个模型在单一且冗长的提示词中试图完成所有事情。你可能会预料到使用量会略有上升，毕竟每个任务更多的调用次数是这种架构的代价，没人会否认这一点。但令我们惊讶的是，对于功能上完全没有变化的任务，使用量竟然翻了三倍。

The intuition that stopped working

失效的直觉

If you’ve only ever run a single LLM call per task, your mental model of cost is roughly linear: longer input, longer output, more tokens, more spend. You can eyeball it and be close enough. That intuition falls apart the moment you introduce orchestration, and it falls apart quietly, which is worse than falling apart loudly. 如果你之前只运行过每个任务单次 LLM 调用，那么你的成本心智模型大致是线性的：输入越长、输出越长，Token 越多，花费就越多。你可以凭直觉估算，且误差不会太大。但一旦引入编排（Orchestration），这种直觉就会失效，而且是悄无声息地失效，这比显而易见的故障更糟糕。

A supervisor agent now makes a decision about what happens next, and that decision itself costs tokens. Each sub-agent carries its own system prompt, its own tool schemas, often its own copy of context the previous agent already had. None of this shows up as “more work being done”; from the user’s side the task is still the same task. It shows up as more machinery wrapped around the task, and machinery isn’t free just because it’s invisible to whoever’s using the product. 现在，一个监督智能体负责决定下一步做什么，而这个决定本身就需要消耗 Token。每个子智能体都带有自己的系统提示词、工具模式（Schema），通常还包含前一个智能体已经拥有的上下文副本。从用户角度看，任务还是那个任务，这些都不会表现为“完成了更多工作”。它表现为围绕任务构建了更多的“机器”，而这些机器并不会因为对产品使用者不可见就变得免费。

We hadn’t budgeted for it. Our architecture reviews had focused on decomposition quality and correctness—did each agent do its job well, did the handoffs make sense—and not once, as far as I can remember, on what the token graph would actually look like once it was live and taking real traffic. That’s on us, and it’s probably on most teams making this move for the first time. 我们没有为此做好预算。我们的架构评审一直专注于分解质量和正确性——每个智能体是否出色地完成了工作，交接是否合理——据我记忆，我们从未考虑过当系统上线并承载真实流量时，Token 的消耗图谱会是什么样子。这是我们的失职，也可能是大多数首次进行此类架构迁移的团队都会犯的错误。

Chasing the wrong suspect

追查错误的嫌疑人

My first assumption was fan-out. Somewhere, the supervisor was spawning more sub-agent calls than the task genuinely needed, and trimming the branching logic would bring the number back down. It felt like the obvious place to look, more agents, more calls, simple arithmetic. So I spent the better part of a day instrumenting call counts per supervisor decision, expecting to catch some node calling three agents when one would have done the job. 我的第一个假设是“扇出”（Fan-out）问题。监督智能体在某个地方触发了比任务实际需求更多的子智能体调用，只要精简分支逻辑，数字就会降下来。这看起来是最显而易见的排查方向：更多的智能体意味着更多的调用，简单的算术题。于是，我花了大半天时间对每次监督决策的调用次数进行埋点，期望抓到某个节点在只需一个智能体就能完成任务的情况下，却调用了三个。

I found almost nothing. The fan-out was doing roughly what it was meant to do, branch for branch. That was frustrating in the specific way that being wrong about an obvious answer is frustrating. You’ve burned a day, and you’re back where you started with one hypothesis eliminated and no replacement. 我几乎一无所获。扇出逻辑在每个分支上都按预期运行。这种挫败感很特别：当你对一个显而易见的答案判断错误时，那种感觉非常糟糕。你浪费了一天时间，却回到了原点，排除掉了一个假设，却没有找到替代方案。

The replacement turned up almost by accident, while I was reading through raw call logs looking for anything unusual rather than testing a theory. One agent’s output would fail a validation step, a malformed tool call, a schema mismatch, occasionally a model deciding to explain itself in prose instead of just calling the tool—and the retry wrapper would silently re-invoke that agent. Silently, because it completed successfully on the retry and nothing downstream ever saw a failure. 替代方案几乎是偶然发现的。当时我正在阅读原始调用日志，试图寻找任何异常情况，而不是在验证某个理论。我发现某个智能体的输出未能通过验证步骤——可能是格式错误的工具调用、模式不匹配，或者模型偶尔决定用自然语言解释自己而不是直接调用工具——而重试包装器（Retry wrapper）会静默地重新调用该智能体。之所以是“静默”，是因为重试后它成功完成了任务，下游从未感知到任何失败。

Which sounds fine, until you notice what re-invoking that agent actually required: rebuilding its upstream context, which in some paths meant re-running a sub-agent that had already produced a perfectly good result, purely to reconstruct the input the failing agent needed. One validation failure, three layers deep in the graph, could trigger a cascade that re-ran work that had never been broken. 这听起来没问题，直到你注意到重新调用该智能体实际上需要什么：重建其上游上下文。在某些路径中，这意味着要重新运行一个已经产生完美结果的子智能体，仅仅是为了重构失败智能体所需的输入。图谱中三层深处的一个验证失败，可能会引发连锁反应，导致重新运行那些原本没有问题的任务。

Nothing about this threw an error a human would see. The pipeline completed and the answer that came back was correct. The retry logic was doing precisely what it had been told to do, retry on failure, it simply had no concept of how expensive the thing it was retrying actually was, or whether re-running already-correct upstream work was necessary at all. 这一切都没有抛出人类可见的错误。流水线完成了，返回的答案也是正确的。重试逻辑完全按照指令行事——失败即重试——它根本没有意识到它所重试的任务有多昂贵，或者是否有必要重新运行已经正确的上游工作。

If you’re building retry logic for an agent graph rather than a single call, this is the part worth sitting with: a retry decorator that behaves perfectly well wrapped around one LLM call becomes a cost amplifier the moment it sits inside a graph with shared upstream state. 如果你正在为智能体图谱而不是单次调用构建重试逻辑，那么这一点值得深思：一个在单次 LLM 调用中表现良好的重试装饰器，一旦置于具有共享上游状态的图谱中，就会瞬间变成成本放大器。

The fix that looked right and wasn’t enough

看起来正确但并不够的修复方案

The obvious response to “retries are expensive” is to cap retry counts and add exponential backoff. We tried it but it only helped a little. 针对“重试成本高昂”这一问题的显而易见的回应是：限制重试次数并增加指数退避（Exponential backoff）。我们尝试了，但收效甚微。

from tenacity import retry, stop_after_attempt, wait_exponential

@retry(
    stop=stop_after_attempt(3),
    wait=wait_exponential(multiplier=1, min=2, max=10),
    reraise=True,
)
def call_agent(agent, payload):
    return agent.invoke(payload)

It’s not the wrong thing to have and you should have retry caps regardless of anything else in this article but it didn’t touch the actual problem. Every retry, capped or not, was still paying full price on the most capable, most expensive model available in the graph, regardless of what had actually gone wrong. A malformed tool call from a summarisation step doesn’t need the same firepower to fix as a genuinely hard reasoning failure. Treating every retry as equally deserving of your strongest model is the mistake, and it’s the one I see recommended most often: add backoff, add a retry limit, move on. 拥有这些机制并没有错，无论本文提到什么，你都应该设置重试上限，但这并没有触及问题的核心。每一次重试，无论是否有限制，都在为图谱中最强大、最昂贵的模型支付全额费用，而不管实际出了什么问题。总结步骤中一个格式错误的工具调用，并不需要像处理真正困难的推理失败那样动用同样的“火力”。将每一次重试都视为值得使用最强模型来处理，这就是错误所在，而这恰恰是我最常看到的建议：增加退避、增加重试限制，然后就此作罢。