Do AI Agents Understand Computer Architecture?

AI 智能体真的理解计算机架构吗？

Abstract: Agents are increasingly asked to design hardware, and increasingly reported to succeed. Such reports establish that a design improved; they cannot establish why. An agent that improves an accelerator may be reasoning about the machine, or may be searching competently over knobs whose meaning it never recovers — and only the first transfers to the next architecture. Existing evaluations cannot tell the two apart, because they vary the agent while holding the framing of the problem fixed.

摘要： 人们越来越多地要求智能体进行硬件设计，且相关成功案例的报道也日益增多。这些报告证实了设计确实得到了改进，但无法解释其背后的原因。一个改进了加速器的智能体，可能是在对机器进行逻辑推理，也可能只是在盲目地搜索那些它从未理解其含义的参数旋钮——而只有前者具备迁移到下一代架构的能力。现有的评估方法无法区分这两者，因为它们在保持问题框架不变的情况下，仅仅改变了智能体本身。

We do the opposite. AutoTuring hands the same agent the same 15-dimensional accelerator space twice: once as named architectural knobs with simulator counters, once as anonymous variables on [0,1], with the evaluator, the legal space and the reachable optima held identical, so that the only thing that varies is whether the problem means anything. The gap between the two is the measurement.

我们采取了相反的做法。AutoTuring 将同一个 15 维加速器空间两次交给同一个智能体：第一次是以带有模拟器计数器的命名架构旋钮形式，第二次是以 [0,1] 区间内的匿名变量形式。在评估器、合法空间和可达最优解保持一致的前提下，唯一的变化就是问题本身是否具有实际意义。这两者之间的差距即为我们的衡量指标。

On a nine-kernel FP16 GEMM basket, meaning pays: the architect beats a modeled H200 by 5.4% and its blind counterpart by 12.3% on average, with 70.1% fewer simulator calls. It does not pay uniquely: a critic loop recovers most of that gap for the blind agent and buys the architect nothing, so architectural knowledge and structured critique behave as substitutes rather than as complements. We report these as preliminary findings — five to six runs per condition on a single modeled accelerator — and take the comparison itself, not the accelerator, to be the contribution.

在包含九个内核的 FP16 GEMM 任务集中，理解含义确实有价值：具备架构知识的智能体平均比建模的 H200 性能高出 5.4%，比盲目搜索的对应版本高出 12.3%，且模拟器调用次数减少了 70.1%。但这种优势并非独有：引入评论循环（critic loop）可以弥补盲目智能体的大部分差距，却无法为具备架构知识的智能体带来额外提升。因此，架构知识与结构化评论在功能上表现为替代关系，而非互补关系。我们将这些作为初步研究结果进行报告——在单个建模加速器上每个条件下运行五到六次——并将这种比较方法本身，而非加速器设计，视为本研究的主要贡献。