Embedded AI evaluators won’t be an easy solution, Hugging Face’s policy chief says

AI labs have staked their reputations on model benchmarking — the supposedly neutral and accurate system of evaluations that are designed to measure whether one AI product is better than another. However, the methods behind these evaluations are complicated and still maturing, said Irene Solaiman, Hugging Face’s chief policy officer, during a discussion at TechCrunch Disrupt last week.

人工智能实验室将自己的声誉押在了模型基准测试上——这是一种被认为中立且准确的评估系统，旨在衡量某款人工智能产品是否优于另一款。然而，Hugging Face 首席政策官艾琳·索莱曼（Irene Solaiman）在上周的 TechCrunch Disrupt 大会讨论中表示，这些评估背后的方法非常复杂，且仍处于成熟阶段。

Testing needs to come together into a more consistent science if the industry is to rely on them in the future. With that in mind, can we trust the current tests that are being released? “Trust is such a weighted term,” said Solaiman. “In terms of whether we can trust results, one big part [of that] is can you reproduce the results of an evaluation? Do you have the level of access that you need to the model itself — this is where open weights come in in a big way,” she added.

如果行业未来要依赖这些测试，它们就需要整合成为一门更具一致性的科学。考虑到这一点，我们能信任目前发布的测试吗？“信任是一个非常有分量的词，”索莱曼说。“就我们能否信任结果而言，其中一个重要部分是：你是否能复现评估结果？你是否拥有对模型本身所需的访问权限——这就是开放权重发挥重要作用的地方，”她补充道。

Reproducibility in model evaluations has long remained one of the stickier aspects of benchmarking — and remains a topic of significant debate in the industry. Outside of an individual test’s validity, there’s also the added question of how that test compares to other evaluations. “They are an evolving science — where we don’t know how to meaningfully compare them to each other,” Solaiman said. “They start to get ineffective relatively quickly, and we don’t have as much of a thriving ecosystem as we need,” she added.

模型评估中的可复现性长期以来一直是基准测试中最棘手的问题之一，也是业内争论的焦点。除了单个测试的有效性之外，还有一个额外的问题，即该测试如何与其他评估进行比较。“它们是一门不断发展的科学，我们还不知道如何有意义地将它们相互比较，”索莱曼说。“它们很快就会变得无效，而且我们还没有形成我们所需要的繁荣生态系统，”她补充道。

“There’s this hunger for an embedded evaluator ecosystem now, but the qualifications that determine who is an effective evaluator across what tasks need a lot more standardization.” The idea of embedded evaluators — in which supposedly “neutral” third-party observers are placed inside AI organizations — has been popularized over the last several months, as AI labs grapple with a rash of safety incidents that have spurred concerns over the potential harms of their technology.

“现在人们渴望建立一个嵌入式评估者生态系统，但确定谁在哪些任务上是有效评估者的资格标准，还需要更多的标准化。”嵌入式评估者的概念——即所谓的“中立”第三方观察员被安置在人工智能组织内部——在过去几个月中变得流行起来，因为人工智能实验室正努力应对一系列安全事件，这些事件引发了人们对其技术潜在危害的担忧。

Hugging Face was famously the target of one of those incidents. Instead of active government regulation to identify and mitigate model safety concerns, labs have pushed for the third-party solution. However, doubts remain over how neutral or effective those evaluators will actually be. Anthropic’s decision to hire Accenture as its first evaluator spurred a good amount of backlash, with some critics mocking the idea that the consulting firm was qualified to do the task at hand.

Hugging Face 曾是此类事件的目标之一。实验室没有寻求积极的政府监管来识别和缓解模型安全问题，而是推动了第三方解决方案。然而，对于这些评估者实际上会有多中立或有效，人们仍存疑虑。Anthropic 决定聘请埃森哲（Accenture）作为其首位评估者引发了相当大的反弹，一些批评人士嘲讽这家咨询公司是否有资格完成这项任务。

Evaluations are now moving from merely a test of the model itself to a test of the entire system around the model — including the harness and agent. The harness — or the tool the user is using to access the model — specifically, has a lot to do with a model’s performance, and a weaker model with a better harness could easily out perform a frontier model, Soleiman shared.

评估工作现在正从单纯的模型测试转向对模型周围整个系统的测试——包括工具链（harness）和智能体（agent）。索莱曼分享道，工具链（即用户访问模型所使用的工具）与模型的性能有很大关系，一个较弱的模型如果配备了更好的工具链，很容易就能胜过前沿模型。

“Agentic evals are really hard and really expensive,” said Solaiman. “We have just more evaluations for models. … We don’t have as many evaluations for harnesses, and we are developing better evaluations for the holistic agent — because that’s what a lot of folks are getting value out of,” she added. In other words: the evaluation space is a bit of a free-for-all at the moment, and if benchmarking is going to continue to be seen as a useful tool for industry leaders, a more uniform, industry-wide approach — where everyone is on the same page — may be necessary.

“智能体评估确实非常困难且昂贵，”索莱曼说。“我们目前有更多的模型评估……但针对工具链的评估还不够多，我们正在为整体智能体开发更好的评估方法——因为这是许多人获得价值的地方，”她补充道。换句话说：目前的评估领域有点混乱，如果基准测试要继续被视为行业领导者的有用工具，那么可能需要一种更统一、全行业通用的方法，让所有人达成共识。