Your agent picks one of two options. Can you test that choice?

你的智能体在两个选项中做出选择。你能测试这个选择吗？

Your agent picks one of two options. Can you test that choice? There’s a moment in almost every agent loop where two options are both defensible and one of them has to win. Two migration plans. Two refactors. Two diagnoses. Two vendors. Two drafts. You put both in the prompt, ask for a pick, and get back something confident. Then you ship it, and you have no idea whether that choice was well-founded or whether the model just happened to prefer the second thing it read. 你的智能体在两个选项中做出选择。你能测试这个选择吗？在几乎每个智能体循环中，总有一个时刻，两个选项都言之有理，但必须选出一个。比如两个迁移方案、两次重构、两个诊断结果、两个供应商或两份草稿。你把两者都放入提示词中，要求它做出选择，并得到一个自信的回答。然后你将其发布，却根本不知道这个选择是有理有据的，还是模型仅仅因为碰巧更偏好它读到的第二个选项。

I’ve been building around this problem for a while. Here’s the part that took me too long to notice: Prose judgements can’t be tested. When the judgement lives inside a prompt, it isn’t a component — it’s a paragraph. You can’t assert on it, you can’t log it, you can’t regression-test it, and you can’t tell a confident pick from a coin flip. When it’s wrong, you find out in production, and what you learn is “the agent chose B” — not how close the call was. 我一直在围绕这个问题进行构建。我花了太久才意识到这一点：散文式的判断是无法测试的。当判断存在于提示词中时，它不是一个组件，而是一段文字。你无法对其进行断言、记录日志、进行回归测试，也无法区分它是自信的选择还是抛硬币的结果。当它出错时，你是在生产环境中发现的，而你所了解的仅仅是“智能体选择了 B”，而不是这个选择有多么勉强。

Worse, there’s a well-known failure mode hiding there: a model asked to pick between A and B and report its own certainty will usually do both in the same breath. Ask it “which is better, and how sure are you?” and you get an answer plus a number that was generated to sound consistent with it. That number is not a measurement of anything. 更糟糕的是，这里隐藏着一种众所周知的故障模式：当模型被要求在 A 和 B 之间做出选择并报告其确定性时，它通常会同时完成这两件事。问它“哪个更好，你有多确定？”，你会得到一个答案，以及一个为了听起来与答案一致而生成的数字。那个数字并不代表任何实际的度量。

Treat the decision as a tool call

将决策视为工具调用

The reframe that helped me: stop asking the agent to have an opinion and give it something to call. One tool, narrow contract: 对我有所帮助的重构思路是：停止要求智能体发表意见，而是给它一个可以调用的工具。一个工具，一个狭窄的契约：

decide(
  task: "Pick a launch date", // stated neutrally, no preferred answer
  option_a: "Ship now — <the case for it>",
  option_b: "Wait two weeks — <the case for it>"
)
and back:
better_option: "A" | "B"
confidence: "83.3%" // how far apart the two were judged
reason: "<why>"

Two properties matter more than the tool itself: 比起工具本身，有两个属性更为重要：

It’s a separate call, not an aside. The judgement is produced in its own request, so it isn’t entangled with the text the agent was already committed to writing.
这是一个独立的调用，而不是附带说明。 判断是在其自身的请求中产生的，因此它不会与智能体已经承诺要编写的文本纠缠在一起。
The confidence is a property of the comparison, not of the answer. It’s the service’s own reading of how far apart the two options were — the thing you actually need for a gate. That distinction is worth being pedantic about, because it changes what you can build on top:
置信度是比较的属性，而不是答案的属性。 它是服务自身对两个选项差距的解读——这正是你进行门控（gate）所真正需要的。这种区别值得我们吹毛求疵，因为它改变了你可以在其之上构建的内容：

confidence >= your_threshold → take the pick automatically
below it → escalate, gather more evidence, or ask a human
置信度 >= 你的阈值 → 自动采纳选择
低于阈值 → 升级处理、收集更多证据或询问人类

The threshold is yours. Nobody else can set it, because it depends on what a wrong call costs you. 阈值由你决定。其他人无法设定，因为它取决于错误决策对你造成的代价。

What this looks like in practice

在实践中是怎样的

If you want to poke at it without writing a client, there’s a public MCP server at https://mcp.turingcorp.net/mcp (discovery is open — tools/list works with no credentials). 如果你想在不编写客户端的情况下进行尝试，有一个公共的 MCP 服务器位于 https://mcp.turingcorp.net/mcp（发现功能是开放的，tools/list 无需凭据即可工作）。

Calling it needs a credential (a 7-day pass, self-service), and the tool returns both a human-readable block and a machine-readable one so you can branch on a code rather than parse prose. 调用它需要凭据（自助获取 7 天通行证），该工具会返回人类可读和机器可读两个区块，这样你就可以根据代码进行分支处理，而不是解析散文。

That last bit is a pattern I’d recommend regardless of whose judge you call: make failures branchable. If your agent can only tell “something went wrong” from a sentence, it can’t decide to refresh a credential or top up a balance on its own. 最后一点是我无论你调用谁的判断器都推荐的模式：让失败变得可分支化。如果你的智能体只能从句子中判断出“出错了”，它就无法自行决定刷新凭据或充值余额。

The honest caveats

诚实的注意事项

I’d rather you know these up front than discover them at 2am. 我宁愿你提前知道这些，而不是在凌晨两点才发现。

No idempotency key yet. The tool declares idempotentHint: false, which is the spec’s way of saying: a client that times out and retries may be charged twice. If your agent retries aggressively, put your own guard in front of it. This is on the roadmap, not done.
暂无幂等键。 该工具声明 idempotentHint: false，这是规范中表示“超时并重试的客户端可能会被重复计费”的方式。如果你的智能体重试频率很高，请在它前面加上你自己的防护措施。这在路线图中，但尚未完成。
No SLA. There’s no availability commitment, and you shouldn’t infer one. It is not an autopilot. It returns a pick and a comparison distance. Whether you act on it, at what threshold, with what review — that’s your policy, and it stays yours.
无 SLA（服务等级协议）。 没有可用性承诺，你不应推断有此承诺。它不是自动驾驶仪。它只返回一个选择和比较距离。你是否根据它采取行动、在什么阈值下、经过什么审查——这是你的策略，且始终属于你。
For high-stakes or irreversible calls, apply your own review process.
对于高风险或不可逆的调用，请应用你自己的审查流程。
The confidence is a reference, not a prediction. High confidence means the two options were judged clearly apart. It does not mean the choice will turn out well.
置信度是参考，而非预测。 高置信度意味着两个选项被判定为有明显差距，但这并不意味着选择的结果一定会好。

Why not just use a second prompt?

为什么不直接使用第二个提示词？

Fair question, and if your volumes are tiny and your stakes are low, a second prompt may be all you need. What you get from pulling it out into a tool is the same thing you get from pulling anything out into a function: a name, a contract, a place to put a test, and a number you can plot. Whether that’s worth a network hop is your call — it depends on whether you intend to gate on the result or just glance at it. 这是一个好问题。如果你的数据量很小且风险很低，第二个提示词可能就足够了。将其提取为工具的好处与将任何东西提取为函数的好处是一样的：一个名称、一个契约、一个放置测试的地方，以及一个你可以绘制图表的数字。这是否值得一次网络跳转由你决定——这取决于你是打算根据结果进行门控，还是仅仅看一眼。

If you want to look at real recorded calls before deciding — actual tasks, both options, the pick, the confidence, and the reasoning — there are 27 of them, published as a dataset: TuringCorp/poe-decider-recorded-cases. They’re recordings, not cherry-picked demos, and they include the low-confidence calls where the two options were close. 如果你想在决定之前查看真实的记录调用——包括实际任务、两个选项、选择结果、置信度和推理过程——这里有 27 个案例，已作为数据集发布：TuringCorp/poe-decider-recorded-cases。它们是真实记录，而非精选演示，其中包含了两个选项非常接近的低置信度调用。

The server is net.turingcorp.mcp/decider in the official MCP registry. Published accuracy by confidence band, with the measurement protocol and failure disclosure, is at api.turingcorp.net — including how it was run rather than just the number. 该服务器在官方 MCP 注册表中为 net.turingcorp.mcp/decider。按置信度区间发布的准确率，以及测量协议和故障披露，可在 api.turingcorp.net 查看——包括它是如何运行的，而不仅仅是数字。