We audited 110 AI usage tools. Here is where the numbers go wrong.

我们审计了 110 款 AI 用量工具，以下是数据出错的地方。

AI products are moving from flat subscriptions to usage-based and outcome-based billing: per-resolution support agents, premium request quotas, per-token gateway billing. Every one of those numbers is produced by a meter, and the meter is usually the vendor’s own. We spent the last month auditing the open-source side of that world — 110 tools that count tokens, track costs, or enforce budgets — and this post is the field report. AI 产品正从固定订阅模式转向基于用量和结果的计费模式：按解决问题的次数收费的客服代理、高级请求配额、按 Token 计费的网关。每一个数字都由计量器产生，而这些计量器通常由供应商自己提供。过去一个月，我们审计了该领域开源侧的 110 款工具（包括计算 Token、追踪成本或执行预算的工具），本文即为此次审计的实地报告。

The short version: 45+ verified bugs, five recurring families, 23 fixes already merged upstream (including langfuse at 34k stars and codeburn at 11k). An independent auditor reproduced our entire conformance suite and confirmed one cache-accounting path that under-reports usage by roughly 99%. Every claim below links to a reproducible check. 简而言之：我们发现了 45 个以上的已验证漏洞，归纳为五大类，其中 23 个修复程序已合并到上游（包括拥有 3.4 万星标的 Langfuse 和 1.1 万星标的 Codeburn）。一位独立审计师复现了我们整套一致性测试，并确认了一条缓存核算路径的用量统计偏差高达 99%。下文中的每一项声明都附有可复现的检查链接。

The five bug families

五大漏洞类别

Stale pricing tables. The most common real bug, and the most boring: pricing tables that lack the current generation of models. In one batch of seven tools, five shipped outdated or missing rate rows. The tool computes confidently from numbers that stopped being true months ago — and every downstream total inherits the error.
过时的定价表。 这是最常见且最无聊的实际漏洞：定价表中缺少当前一代的模型。在一组 7 款工具中，有 5 款内置了过时或缺失的费率行。工具基于数月前已失效的数据进行自信计算，导致所有下游统计结果都继承了这一错误。
Provider-wrong cache multipliers. Cache reads and writes are billed at different multipliers per provider. A common shortcut hardcodes one provider’s ratios and applies them to everyone. We found a tool applying Anthropic’s cache-read discount to OpenAI models, underestimating cache reads by 5x. The billing math is wrong in a way that looks plausible on every individual line.
供应商缓存乘数错误。 不同供应商对缓存读写的计费乘数不同。一种常见的捷径是硬编码某一家供应商的比例并将其套用到所有供应商身上。我们发现某款工具将 Anthropic 的缓存读取折扣应用到了 OpenAI 模型上，导致缓存读取量被低估了 5 倍。这种计费逻辑的错误在于，它在每一行明细上看起来都“合情合理”。
Retry double-counting in stream aggregation. When a streaming request is retried byte-identical, some aggregators count both attempts. In one public corpus of 604 re-emitted events — 46% of the corpus — the bytes were identical, and naive aggregation doubled them. Conversely, tools that deduplicate too aggressively miss genuine retried work. Both directions lose money for someone.
流聚合中的重试重复计算。 当流式请求进行字节完全相同的重试时，一些聚合器会同时计算两次尝试。在一个包含 604 个重发事件的公共语料库中，46% 的事件字节完全相同，而简单的聚合方式将它们翻倍计算了。相反，过度去重的工具则会漏掉真实的重试工作。这两种情况都会导致某一方损失金钱。
Absent treated as zero. When a usage field is missing, or 0 turns “unknown” into “free”. It reads like defensive coding and behaves like a discount nobody authorized. The fix is semantic, not syntactic: absent must stay absent, and rollups must be allowed to say UNPROVABLE instead of zero.
将“缺失”视为“零”。 当用量字段缺失时，或者将“未知”转变为“免费”。这看起来像是防御性编程，但实际表现却像是未经授权的折扣。修复方法在于语义而非语法：缺失必须保持缺失，汇总结果应允许显示“无法证明（UNPROVABLE）”而非零。
Window boundary errors. Quota windows anchored to wall-clock time with fixtures pinned to absolute dates — the tool passes tests for 30 days and then fails everywhere at once. We know this one intimately because our own CI did it to us.
窗口边界错误。 配额窗口锚定在挂钟时间，且固定在绝对日期上——工具在 30 天内测试通过，随后在所有地方同时失效。我们对此深有体会，因为我们自己的 CI 系统就曾犯过这种错。

An independent reproduction

独立复现

The part we are proudest of is not our numbers — it is that an external auditor, working separately, reproduced our full conformance suite (236 checks) and then contributed his own quantified audit of a commercial provider’s cache accounting: four code paths failing, usage under-reported by 98.9% and 95.1% on the affected paths, pinned to an exact commit. That report is now in our evidence tree with named credit. A third party spending real effort to check your work is worth more than any testimonial. 我们最自豪的不是我们的数据，而是一位外部审计师独立复现了我们整套一致性测试（236 项检查），并贡献了他对某商业供应商缓存核算的量化审计：发现 4 条代码路径存在故障，受影响路径的用量统计分别被低估了 98.9% 和 95.1%，并精确到了具体的提交记录。该报告现已列入我们的证据树并署名致谢。第三方投入实际精力来核查你的工作，其价值远胜于任何推荐信。

What merged upstream

已合并至上游的内容

23 fixes across the ecosystem carry the audit’s fingerprints — pricing table updates, cache multipliers corrected per provider, retry-collapse guards with regression fixtures, absent-vs-zero semantics made explicit. None of these were adversarial finds. Every one started as a thread where the maintainer was already discussing the problem, which is the only place we work. 整个生态系统中 23 个修复程序都带有此次审计的印记——包括定价表更新、按供应商修正缓存乘数、带有回归测试的重试折叠保护、明确的“缺失 vs 零”语义。这些发现并非对抗性的，每一个修复都始于维护者已经在讨论该问题的线程，这也是我们唯一的工作方式。

What vendors should publish

供应商应发布的内容

We checked 20 commercial vendors for one thing: a published process for what happens when the meter is wrong — a dispute path, a correction policy, anything. None of these had one. When the vendor runs the meter, grades its own homework, and publishes no correction process, “trust us” is the whole control framework. Two minimum standards would change that: a named dispute path, and machine-checkable billing disclosures. 我们检查了 20 家商业供应商，只为寻找一件事：当计量出错时，是否有公开的处理流程——比如争议渠道、纠错政策等。结果一家都没有。当供应商既负责计量，又负责给自己打分，且不发布任何纠错流程时，“请信任我们”就成了唯一的控制框架。两个最低标准可以改变现状：明确的争议渠道和机器可校验的计费披露。

Verify your own bill in ten minutes

十分钟内验证你的账单

The conformance pack is open source (MIT). Export your usage data, run the checker locally — nothing leaves your machine — and it separates logical operations from physical attempts, cache reads from cache writes, and absent from zero, with every verdict traceable to a named rule. If a number on your invoice survives an independent recount, you can defend it. If it doesn’t, now you have the receipt. 该一致性测试包是开源的（MIT 协议）。导出你的用量数据，在本地运行检查器（数据不会离开你的机器），它会将逻辑操作与物理尝试、缓存读取与缓存写入、缺失与零区分开来，每一项结论都可以追溯到具体的规则。如果账单上的数字经得起独立重算，你就可以捍卫它；如果经不起，现在你手里就有证据了。

The full audit report with per-tool findings: campaigns/audit-report-2026-09.md. The settlement spec behind the checker: AMS-1. 完整的审计报告及各工具的发现：campaigns/audit-report-2026-09.md。检查器背后的结算规范：AMS-1。