The LLM Judge That Kept Agreeing With Itself

那个不断自我肯定的 LLM 裁判

What a production incident taught me about trusting a model to judge another model’s work 一次生产事故教会我：为何不能盲目信任由模型来评估另一个模型的工作

We had a multi-agent pipeline where one agent generated a SQL query from a user’s natural-language question and a second agent — “the judge” — decided whether that query was safe and correct enough to execute automatically or whether it needed to stop and wait for a human to confirm before running. 我们曾构建了一个多智能体流水线：第一个智能体负责根据用户的自然语言问题生成 SQL 查询，而第二个智能体（即“裁判”）则负责判断该查询是否足够安全且准确，从而决定是自动执行，还是暂停并等待人工确认。

For a few weeks it worked well enough that we stopped watching it closely. Then a query that should have been flagged got approved and ran. Nothing catastrophic happened, no data was lost, no write went to the wrong table, but the returned result was completely wrong. 在最初的几周里，它运行得还算不错，以至于我们放松了警惕。直到有一天，一个本该被拦截的查询却被批准并执行了。虽然没有发生灾难性的后果，没有数据丢失，也没有写入错误的表，但返回的结果完全是错误的。

This happened because the query had silently dropped a filter clause the user’s question clearly implied and it took us a confused half-hour with an analyst to work out that the number they’d been given was simply incorrect. Just wrong and confidently presented as right by a judge that had approved it without hesitation. 事故的原因是：该查询悄无声息地丢弃了一个用户问题中明确隐含的过滤条件。我们和分析师困惑了半小时，才发现给出的数据完全不对。这不仅是错误，而且是由一个毫不犹豫地批准了该查询的“裁判”自信满满地呈现出来的。

That’s the incident that made us stop treating “LLM judge approved it” as equivalent to “it’s correct” and start treating the judge itself as a component that needed its own testing. 正是这次事故让我们意识到，不能再将“LLM 裁判已批准”等同于“结果正确”，我们必须将裁判本身视为一个需要独立测试的组件。

The incident and the first assumption

事故与最初的假设

My first assumption was that the judge had made a one-off mistake. LLMs aren’t perfectly reliable, a bad call happens, but you move on. So I re-ran the same query and the same judge prompt against it in isolation expecting either a repeat of the mistake or a correction. It approved the query again with the same missing filter and same confidence. 我最初的假设是裁判犯了一个偶然的错误。LLM 并非绝对可靠，偶尔误判在所难免，修正即可。于是，我将同样的查询和同样的裁判提示词（prompt）单独重新运行，预期它要么重复错误，要么进行修正。结果，它再次批准了该查询，依然漏掉了过滤条件，且表现得同样自信。

Whatever was happening wasn’t a random miss, it was a pattern the judge would reproduce reliably which meant something structural was going in the wrong direction. To get to the roots of the issue I pulled a batch of past judge decisions and started comparing them against what a human reviewer would have said about the same queries and that’s when the pattern got specific enough to name. 无论发生了什么，这绝非随机失误，而是一种裁判能够稳定复现的模式，这意味着系统结构出了问题。为了找到问题的根源，我调取了一批过去的裁判决策，并将它们与人工审核员对相同查询的评估结果进行对比，这时，这种模式终于清晰到可以被定义了。

What the judge was actually doing

裁判到底在做什么

The generator agent and the judge agent were built on the same underlying model. That wasn’t a design choice but it was just the model we’d standardised on for cost reasons across the whole pipeline. When I swapped in queries generated by a different model with the same task and schema, comparable quality on manual inspection, the judge got noticeably stricter. It caught issues in the other model’s queries that it had been letting through in its own model’s queries. 生成智能体和裁判智能体是基于同一个底层模型构建的。这并非刻意设计，只是出于成本考虑，我们在整个流水线中统一使用了该模型。当我换用另一个模型生成任务和模式相同、经人工检查质量相当的查询时，裁判明显变得严格了。它捕捉到了其他模型查询中的问题，而这些问题在它“自己人”的查询中却被放行了。

That’s self-preference bias, and it’s a documented behaviour in LLM-as-judge research, not something unique to our setup. What I hadn’t appreciated until it caused an actual incident is how consistent the effect can be. This wasn’t a judge that was randomly generous. It was a judge that was specifically generous toward outputs written in a style and structure it recognised as its own. 这就是“自我偏好偏差”（Self-preference bias），这是 LLM 作为裁判的研究中记录在案的行为，并非我们系统独有。在事故发生前，我没意识到这种效应竟然如此稳定。这并不是一个随机宽容的裁判，而是一个专门对它识别为“自己风格和结构”的输出表现出宽容的裁判。

How a judge lies to you

裁判是如何欺骗你的

Once I found self-preference bias in our judge, I started looking for other systematic ways the judge could be wrong and here are my findings. 在发现裁判存在自我偏好偏差后，我开始寻找裁判可能出错的其他系统性原因，以下是我的发现：

Self-preference bias – An LLM acting as a judge can rate its own outputs, or outputs from its own model family, higher than others as happened in our case. The core driver of this bias is perplexity familiarity. LLMs tend to assign higher scores to text that is more predictable and has lower perplexity, which means it is generated by its own model family. 自我偏好偏差 —— 正如我们遇到的情况，担任裁判的 LLM 可能会给自己的输出或同系列模型的输出打出比其他模型更高的分数。这种偏差的核心驱动力是“困惑度熟悉感”（perplexity familiarity）。LLM 倾向于给那些更具可预测性、困惑度更低的文本打高分，这意味着这些文本是由其同系列模型生成的。
Verbosity bias – When you give a judge two answers to compare in which one is correct and concise and the other is same with extra context, the longer one wins more often than the shorter one. This was another bias pattern which we observed where the judge is giving higher scores to the SQL which contains more detailed comments rather than one which contains just the “exact” query. 冗长偏差 —— 当你让裁判比较两个答案，其中一个简洁正确，另一个包含额外背景信息时，较长的答案往往比短的更容易胜出。我们观察到的另一种偏差模式是：裁判给包含详细注释的 SQL 打分更高，而不是给那些仅包含“精确”查询的 SQL 打分。
Position bias – For pairwise comparisons like “which of these two outputs is better”, the order you present the outputs shifts the outcome of your judge. Swap the order by keeping everything else identical and the judgment might flip. This is very easy to catch; run both orders and see if the verdict holds. 位置偏差 —— 对于“这两个输出哪个更好”之类的成对比较，你呈现输出的顺序会改变裁判的结果。保持其他条件不变，仅交换顺序，判断结果就可能反转。这一点很容易验证：尝试两种顺序，看看结论是否一致。

None of these three invalidate the approach. They simply rule out treating a judge’s score as an objective measurement, the way you’d treat a unit test passing or failing. It’s an opinion from a consistent but biased reviewer. 这三种偏差并不会否定“LLM 裁判”这种方法本身。它们只是排除了将裁判评分视为客观衡量标准（就像单元测试通过或失败那样）的可能性。它仅仅是一个来自“一致但存在偏见”的评审员的观点。