Generating in the Limit with Infinitely Many Hallucinations

在无限幻觉下的极限生成

Abstract: The classic paradigm of language identification in the limit models learning as a game between an adversary, who reveals strings from an unknown target language, and a learner tasked with identifying that language.

摘要： 经典的“极限语言识别”范式将学习建模为一场博弈：对手揭示来自未知目标语言的字符串，而学习者则负责识别该语言。

The recently introduced framework of language generation in the limit shifted the objective to better reflect modern language modeling, requiring the learner to produce valid, unseen strings from the target language.

最近提出的“极限语言生成”框架转变了目标，以更好地反映现代语言建模的需求，即要求学习者从目标语言中生成有效的、未见过的字符串。

Related work highlighted a fundamental tension: a broad coverage of the target often comes at the cost of validity. We introduce a new notion of precision and recast this problem as the classic recall-precision trade-off.

相关研究强调了一个根本性的矛盾：对目标的广泛覆盖往往以牺牲有效性为代价。我们引入了一个新的精确度概念，并将此问题重新定义为经典的召回率与精确度之间的权衡。

We analyze generation in the limit under varying constraints on enumeration, novelty, and validity, aimed at reflecting settings closer to those encountered by large language models.

我们分析了在枚举、新颖性和有效性等不同约束下的极限生成，旨在反映更接近大型语言模型所面临的实际场景。

A key contribution is our analysis of learners that are not eventually valid: we allow infinitely many mistakes, provided their frequency tends to zero so that precision remains one.

本研究的一个关键贡献在于对“非最终有效”学习者的分析：我们允许出现无限多次错误，前提是这些错误的频率趋于零，从而使精确度保持为一。

We show that this relaxation can strictly increase recall when the adversary permanently withholds a large portion of the target language.

我们证明，当对手永久隐瞒目标语言的很大一部分时，这种放宽条件可以严格提高召回率。

We also study a continuous relaxation of the novelty constraint that requires only a fixed fraction of outputs to be novel.

我们还研究了新颖性约束的一种连续放宽形式，即仅要求固定比例的输出必须是新颖的。

Taken together, our results move toward a more realistic model of language generation where occasional errors and repetitions are unavoidable, but their rates are controlled.

综上所述，我们的研究结果朝着更现实的语言生成模型迈进了一步，在该模型中，偶尔的错误和重复是不可避免的，但其发生率是可控的。