Artificiality's First Exam: An LLM-Referenced C-test for Supraintelligence

Computer Science > Artificial Intelligence arXiv:2610.14005 (cs) [Submitted on 27 Sep 2026] Title: Artificiality’s First Exam: An LLM-Referenced C-test for Supraintelligence Authors: Jose Hernandez-Orallo.

计算机科学 > 人工智能 arXiv:2610.14005 (cs) [提交于 2026 年 9 月 27 日] 标题：人工性的第一次考试：一种用于超智能的基于大语言模型参考的 C-test 作者：Jose Hernandez-Orallo。

Abstract: Under the accelerated pace of AI development, any benchmark whose difficulty is calibrated with humans or current AI systems becomes circular, or saturates soon. How can we then evaluate machines that may become more capable than the current machines and the people who write the tests?

摘要：在人工智能发展的加速步伐下，任何以人类或当前人工智能系统校准难度的基准测试都会变得循环，或很快达到饱和。那么，我们该如何评估那些可能比当前机器以及编写测试的人类更强大的机器呢？

One answer to the question hinges on inverse problems, inductive inference questions for which difficulty depends on the complexity of the underlying mechanism, given the observations and the previous knowledge. Levin’s Kt, a computable variant of Kolmogorov complexity, puts a theoretical upper bound to that difficulty, balancing the simplest representation encoding for the mechanism and the logarithm of the time steps to run that mechanism.

该问题的一个答案取决于反向问题，即归纳推理问题，其难度取决于在给定观测结果和先验知识的情况下，底层机制的复杂性。Levin’s Kt 是柯尔莫哥洛夫复杂性的一种可计算变体，它为这种难度设定了一个理论上限，平衡了机制的最简表示编码与运行该机制所需时间步长的对数。

Previous tests derived from Levin’s Kt, such as the original C-test, are based on minimalistic Universal Turing Machines (UTM), capable of very limited representations and no previous knowledge. Reasoning large language models (LLM) are UTMs that capture a compendium of human culture as knowledge, using prompts as natural language programs.

先前源自 Levin’s Kt 的测试（例如最初的 C-test）基于极简的通用图灵机 (UTM)，其表示能力非常有限且不具备先验知识。推理大语言模型 (LLM) 是能够将人类文化纲要作为知识捕获的通用图灵机，它们使用提示词作为自然语言程序。

This paper presents the first test based on Levin’s Kt with LLMs as representational machines. Fresh test items can be produced at high difficulties, estimated in a ratio scale measured in bits, before any human or frontier model can solve them.

本文提出了第一个基于 Levin’s Kt 并以大语言模型作为表示机器的测试。在任何人类或前沿模型能够解决这些问题之前，就可以生成高难度的新测试项，并以比特为单位的比例尺度进行估算。

We formalise the measure, devise a lightweight generation and certification pipeline, explore the effect of the power of the reference machine and the gap between examiner and examinee. We produce a first battery of certified items administered to twelve LLMs.

我们对该度量进行了形式化，设计了一个轻量级的生成和认证流程，并探讨了参考机器的能力以及考官与考生之间差距的影响。我们制作了第一批认证测试项，并将其应用于十二个大语言模型。

The resulting agent characteristic curves decrease with difficulty, down to the uncharted high-difficulty region, the supra-intelligence window of items we can certify, yet beyond generations of frontier intelligence.

所得出的智能体特征曲线随难度增加而下降，直至进入未知的超高难度区域——即我们能够认证的超智能窗口，但该区域已超出了当前前沿智能的代际水平。