Cheapest LLM for 77-intent banking classification: GPT-6.1 Terra at $0.000727 a success

在 150 条 BANKING77 消息的测试中，GPT-6.1 Terra 是十个通过 70% 及格线且被证明与基准模型 GPT-6.1 Sol 持平的模型中最便宜的一个。它的任务通过率为 84.0%，而 Sol 为 86.0%；其单次成功任务成本为 0.000727 美元，相比 Sol 的 0.001272 美元降低了 42.8%。

On 150 BANKING77 messages, GPT-6.1 Terra is the cheapest of ten models that clears a 70% pass bar and can be shown to match the baseline, GPT-6.1 Sol. It passes 84.0% of the tasks against 86.0% for Sol, at $0.000727 per successful task against $0.001272: 42.8% less.

有两个更便宜的模型值得直接解读，因为表格中最便宜的一行并不总是最佳选择。GPT-6 Luna 的通过率为 82.0%（区间 75.1-87.3%），单次成功任务成本约为 0.00009 美元，比 Sol 便宜约 14 倍。它达到了 70% 的及格线，但在 150 行的配对测试中，无法证明其与 Sol 的差距在 5 个百分点以内：它在 Sol 通过的任务中失败了 8 次，而在 Sol 失败的任务中通过了 2 次。“达到及格线”和“匹配基准”是不同的主张，而结论中明确的是后者。

Two cheaper models deserve a straight reading, because the cheapest line in a table is not always the pick. GPT-6 Luna passes 82.0% (interval 75.1-87.3%) at $0.00009 per successful task, about 14 times cheaper than Sol. It clears the 70% bar. But on 150 rows the paired test cannot show it is within 5 points of Sol: it failed 8 tasks Sol passed and passed 2 that Sol failed. “Clears the bar” and “matches the baseline” are different claims, and the verdict names the second one.

Claude Haiku 5.5 的通过率为 80.0%，单次成功任务成本为 0.000138 美元，且拥有本次运行中最低的延迟（中位数 249 毫秒）。它同样达到了及格线，但未被证明与 Sol 持平。

Claude Haiku 5.5 passes 80.0% at $0.000138 per successful task with the lowest latency in the run (median 249 ms). It also clears the bar and is not shown to match Sol.

测试运行说明：从 3,080 行的 BANKING77 测试集（CC BY 4.0）中通过固定种子抽取了 150 条消息。任务要求：从 77 个意图标签列表中回复单个标签。温度设置为 0，若回复不是有效标签则最多重试 2 次，计算每次尝试的成本，价格取自 2026 年 11 月 5 日的 OpenRouter。所有十个模型的总支出为 0.58 美元。

The run. 150 messages drawn with a fixed seed from the 3,080-row BANKING77 test split (CC BY 4.0). One task: reply with the single intent label from the 77-label list. Temperature 0, up to 2 retries when the reply is not a valid label, every attempt’s cost counted, prices read from OpenRouter on 2026-11-05. Total spend for all ten models: $0.58.

关于及格线：正确率需达到 70%。只有当模型 95% Wilson 区间的下限达到或超过及格线时，才算“通过”；若区间下限低于及格线则为“临界”；若通过率低于及格线则为“未达标”。“匹配”意味着单侧配对测试显示其“表现不比基准差超过 5 个百分点”的结果成立。

Pass bar. 70% correct. A model “clears” only if the lower end of its 95% Wilson interval is at or above the bar, “borderline” if the interval reaches below it, “below” if the pass rate is under it. “Matches” means an exact one-sided paired test of “no more than 5 points worse than the baseline” succeeds.

为什么代币价格会产生误导？Deepseek-v4.1-flash 虽然达到了及格线，但在 150 个任务中需要 32 次重试（每个任务 1.21 次尝试），且最终答案中有 11 个仍不是有效标签，其总支出的 31.3% 用于重试。其单次成功成本（0.000157 美元）最终高于 Haiku（0.000138 美元）。Qwen3.8-flash 将 19.8% 的资金用于重试，且 95 分位延迟为 6.7 秒。Mistral-large-4 是极端情况：150 个任务中重试 106 次，44.6% 的支出用于重试，单次成功任务成本为 0.002274 美元，是表中通过率 63.3% 的模型中最昂贵的一行。仅计算首次尝试代币的账单会掩盖所有这些问题。

Why a token price misleads. deepseek-v4.1-flash cleared the bar, but it needed 32 retries across 150 tasks (1.21 attempts per task), 11 of its final answers were still not a valid label, and 31.3% of everything it spent went on retries. Its cost per success ($0.000157) ends up above Haiku’s ($0.000138). qwen3.8-flash spent 19.8% of its money on retries and had a 95th-percentile latency of 6.7 seconds. mistral-large-4 is the extreme case: 106 retries across 150 tasks, 44.6% of its spend on retries, and $0.002274 per successful task, the most expensive line in the table at 63.3%. A bill that counts only first-attempt tokens hides all three.

错误是什么样的？对于“卡片送达需要多长时间？”，Terra 回答了 card_delivery_estimate；而数据中的标签是 card_arrival。对于“卡片什么时候到？”，它回答了 card_arrival，而数据中写的是 card_delivery_estimate。两者都是对句子的合理解读。在 77 个标签的任务中，有些错误属于这类标签边界判断问题，这就是为什么“哪个模型最好”的答案取决于你的标签和及格线，而不是排行榜。

What a miss looks like. For “How long does a card delivery take?”, Terra answered card_delivery_estimate; the label in the data is card_arrival. For “When will the card arrive?” it answered card_arrival where the data says card_delivery_estimate. Both are plausible readings of the sentence. On a 77-label task some misses are label-boundary calls like these, which is why the answer to “which model” depends on your labels and your bar, not on a leaderboard.

检查或重新运行。150 行数据、提示词、价格和重新运行脚本都在结论页面上，因此你可以使用自己的 OpenRouter 密钥进行相同的比较（全部十个模型的运行成本为 0.58 美元），看看你的数据结果如何。价格每月变动；页面标注了上次重新核对的日期。想要在自己的数据上得到同样的结论？导入 10-100 对你的输入/预期结果，添加你当前的模型，页面会列出达到你及格线的最便宜模型，并显示单次成功任务成本、重试次数以及它将产生的失败情况。早期访问期间免费，你只需使用自己的密钥支付 OpenRouter 的调用费用。

Check it or re-run it. The 150 rows, the prompt, the prices and the re-run script are on the verdict page, so you can run the same comparison with your own OpenRouter key (the full ten-model run cost $0.58) and see where your numbers land. Prices move monthly; the page notes the day it last re-checked them. Want the same verdict on your own data? Import 10-100 of your input/expected pairs, add your current model, and the page names the cheapest model that clears your bar, with cost per successful task, retries counted, and the failures it will have. Free during early access, you pay OpenRouter for the calls with your own key.