The agent gave the right answer and did the wrong thing

查看原文 / View Original 编辑 / Edit

The agent gave the right answer and did the wrong thing

智能体给出了正确的答案，却做了错误的事情

The bug that passes every test. A refund agent ships v2. A customer asks for a refund. The agent replies: “Your refund of $48.20 has been issued and will appear in 3–5 business days.” That is exactly what v1 said. The amount is right. The tone is right. An LLM judge scores it identically. A regression suite comparing outputs sees no diff. Ship it. 这是一个能通过所有测试的 Bug。退款智能体发布了 v2 版本。一位客户要求退款，智能体回复道：“您的 48.20 美元退款已处理，将在 3-5 个工作日内到账。”这与 v1 版本的回复完全一致。金额正确，语气也正确。大模型（LLM）评测员给出的评分也相同。对比输出的回归测试套件没有发现任何差异。发布吧。

Underneath, this happened: v1 (approved) v2 (shipped) refund.request refund.request -> policy.retrieve -> order.lookup -> order.lookup -> payment.refund -> fraud.check -> payment.refund <- charged twice -> refund.calculate -> customer.notify -> payment.refund -> customer.notify. The refund-policy retrieval is gone. The fraud check is gone. And the payment write happened twice — a timeout, a retry, and the payment service’s idempotency ledger recording both entries. The customer got the right sentence. The company refunded $96.40 and skipped its own compliance checks. Every output-based control in the pipeline was green. 在底层，实际发生的情况是：v1 版本（已批准）与 v2 版本（已发布）的执行路径不同。退款策略检索消失了，欺诈检查也消失了。支付写入操作执行了两次——由于超时触发了重试，支付服务的幂等账本记录了两次条目。客户收到了正确的回复，但公司却退款了 96.40 美元，并跳过了自身的合规检查。流水线中所有基于输出的控制指标全是绿色的。

This is not a hypothetical. It is the failure mode that appears the moment an agent has a retry policy, a tool it can skip, and a prompt somebody edited on a Friday. Output evaluation is structurally blind to it, because the output is fine. The execution trajectory is where the bug lives. And a trajectory is exactly what a distributed trace is. 这不是假设，而是当智能体具备重试策略、可跳过的工具以及某人在周五修改过的提示词（Prompt）时，必然会出现的故障模式。基于输出的评估在结构上对此是盲目的，因为输出本身没问题。Bug 存在于执行轨迹中，而分布式追踪（Distributed Trace）恰恰就是轨迹本身。

FlightRules

FlightRules（飞行规则）

FlightRules turns SigNoz traces into deterministic release contracts. It mines a trajectory contract from runs a human approved, then evaluates every subsequent release against it, and fails the release with the traces that prove it. In CI that is one command: FlightRules 将 SigNoz 的追踪数据转化为确定性的发布契约。它从人类批准的运行记录中挖掘出轨迹契约，然后评估后续的每一次发布，如果发现违规，则会通过追踪记录证明并拦截发布。在 CI 中，这只需要一条命令：

$ flightrules gate check --release refund-agent-v2
FAIL: This release exceeded one or more trajectory thresholds.
Findings:
MISSING_PREREQUISITE (skipped_check) fraud.check is required at least 1 time(s) but occurred 0 time(s).
DUPLICATE_SIDE_EFFECT (duplicate_write) payment.refund occurred 2 times; the contract permits at most 1 per run.
exit code: 2

Exit code 2. The pipeline stops. 退出代码 2。流水线停止。

Architecture

架构

Five stages, and the interesting design decisions are all about what is not allowed in each. 架构分为五个阶段，其中有趣的设计决策在于每个阶段中“不允许做什么”。

  1. Instrument. The demo is a refund agent and five services — policy, orders, fraud, payment, notification — instrumented with OpenTelemetry and exporting traces, metrics and logs over OTLP into SigNoz. The agent emits gen_ai.* attributes for tool identity and operation, plus a small FlightRules namespace: agent.side_effect, agent.data_domain, agent.retry.number, agent.idempotency.present. It emits no prompt, no model output, no tool arguments, no tool results and no chain-of-thought. That was a constraint from the start, and it turned out to be the interesting one: if trajectory enforcement works without any of it, the product is deployable in places prompt logging is not. It does work. The forbidden-key redactor removes those attributes rather than replacing them with [redacted] — a redaction marker would still record that the product collected one.

  2. 埋点（Instrument）。 演示包含一个退款智能体和五个服务（策略、订单、欺诈、支付、通知），它们通过 OpenTelemetry 埋点，并将追踪、指标和日志通过 OTLP 导出到 SigNoz。智能体发出用于标识工具和操作的 gen_ai.* 属性，以及一个小的 FlightRules 命名空间：agent.side_effect、agent.data_domain、agent.retry.number、agent.idempotency.present。它不发出提示词、模型输出、工具参数、工具结果或思维链。这是从一开始就设定的约束，结果证明这非常关键：如果轨迹强制执行在没有这些信息的情况下也能工作，那么该产品就可以部署在不允许记录提示词的环境中。它确实做到了。禁止键脱敏器会直接移除这些属性，而不是用 [redacted] 替换——因为脱敏标记本身也会记录产品曾收集过这些信息。

  3. Retrieve. Everything goes through the SigNoz MCP Server, using the official MCP TypeScript SDK. The first real discovery of the project came here: signoz_get_trace_details cannot return custom span attributes. It returns a fixed column set. Every contract rule that reads an attribute would have silently seen nothing. The path that works is signoz_execute_builder_query with requestType: "raw" and selectFields declaring fieldContext: "tag". There is a further asymmetry that costs an afternoon if you meet it by accident: discovery tools accept “attribute” as an alias for “tag”, but selectFields and groupBy require “tag”.

  4. 检索（Retrieve）。 所有数据都通过 SigNoz MCP 服务器，使用官方的 MCP TypeScript SDK。该项目的第一项重大发现就在这里：signoz_get_trace_details 无法返回自定义的 Span 属性，它只返回固定的列集。任何读取属性的契约规则都会静默地“一无所获”。可行的路径是使用 signoz_execute_builder_query，并将 requestType 设为 "raw"，在 selectFields 中声明 fieldContext: "tag"。这里还有一个容易让人浪费一下午的陷阱：发现工具接受 “attribute” 作为 “tag” 的别名，但 selectFields 和 groupBy 必须使用 “tag”。

  5. Reconstruct. Spans become a causal graph, then a canonical form, then a fingerprint. Volatile identifiers are normalised out, so two runs of the same behaviour with different order IDs produce the same fingerprint. Identical traces produce identical fingerprints — asserted with property tests over reordered input, reordered rules and reordered attribute keys.

  6. 重构（Reconstruct）。 Span 被转化为因果图，然后是规范形式，最后是特征指纹。易变标识符会被归一化处理，因此两次相同行为但订单 ID 不同的运行会产生相同的指纹。相同的追踪产生相同的指纹——这一点通过对重排后的输入、重排后的规则和重排后的属性键进行属性测试得到了验证。

  7. Mine and contract. Twenty-five approved runs collapse into route families. The dominant family is proposed as approved; anything rarer is left pending for a human, because approving every family automatically would defeat the review step the product exists to make explicit. From the approved family the miner proposes a contract in a small YAML DSL with eleven rule types — required spans, forbidden spans, cardinality, required edges, required ancestry, forbidden paths, attribute constraints, allowed values, retry budgets, approved routes, numeric budgets.

  8. 挖掘与契约（Mine and contract）。 25 次批准的运行被归纳为路由族。占主导地位的族被提议为已批准；较罕见的则留给人工审核，因为自动批准所有族会使该产品旨在明确的审查步骤失去意义。从已批准的族中，挖掘器会提出一个小型 YAML DSL 契约，包含 11 种规则类型：必需 Span、禁止 Span、基数、必需边、必需祖先、禁止路径、属性约束、允许值、重试预算、已批准路由、数值预算。

  9. Evaluate and gate. A pure function over a graph and a contract. Then aggregation across a release, then a gate decision, then a process exit code.

  10. 评估与门控（Evaluate and gate）。 这是一个基于图和契约的纯函数。随后进行跨发布的聚合，得出门控决策，最后输出进程退出代码。

The determinism boundary

确定性边界

The single decision everything else follows from: no model, no clock, no network and no randomness participates in a pass/fail. A model may explain a violation to a human. It may never decide whether a rule passed. The consequences show up in odd places: The evaluator reads no clock. The caller injects completedAt, so a timestamp cannot enter the hashed region and two evaluations of the same evidence are byte-identical. Route similarity is a weighted Jaccard ratio compared by exact integer cross-multiplication, never by dividing into a double. A release decision must not depend on binary rounding of a threshold the author wrote in decimal. evaluateRunSafely is the one place an internal error becomes a status, and it structurally cannot produce “pass”. The last one matters more than it sounds. A release gate that can return green after an internal error is worse than no gate, because it launders a crash into a permission. 一切决策都遵循单一原则：模型、时钟、网络和随机性均不参与“通过/失败”的判定。模型可以向人类解释违规原因，但绝不能决定规则是否通过。这一原则的后果体现在一些奇怪的地方：评估器不读取时钟。调用者注入 completedAt，因此时间戳不会进入哈希区域，对同一证据的两次评估在字节上是完全一致的。路由相似度是加权 Jaccard 比率，通过精确的整数交叉乘法进行比较，绝不通过除法转换为浮点数。发布决策绝不能依赖于作者以十进制编写的阈值的二进制舍入。evaluateRunSafely 是内部错误转化为状态的唯一地方，它在结构上不可能产生“通过”的结果。最后一点比听起来更重要：一个在内部错误后仍能返回“绿色”的发布门控，比没有门控更糟糕，因为它将一次崩溃“洗白”成了许可。

Absence is not a value

缺失并非一种值

The subtlest correctness problem in the whole product is what to do when the evidence is missing. Consider agent.idempotency.present on a payment write. If the attribute is absent, has the rule equals: true passed or failed? Neither. It is undecided, and the evaluator says so — insufficient_evidence, and the run reports insufficient_data rather than pass. 整个产品中最微妙的正确性问题是：当证据缺失时该怎么办？考虑支付写入时的 agent.idempotency.present。如果该属性缺失，规则 equals: true 是通过还是失败了？都不是。它是未决状态，评估器会明确指出——insufficient_evidence（证据不足），运行结果报告为 insufficient_data（数据不足）而非通过。

← 返回新闻列表 / Back to news list