Disentangling Statistical Preemption from Entrenchment in Language Models’ Avoidance of Overgeneralization

解构语言模型避免过度概括时统计先占与固化机制的区别

Abstract: How do learners avoid overgeneralizations such as Tom laughed me without explicit negative evidence? Constructionists have posited two proposals that describe indirect negative evidence against overgeneralizations: preemption (which privileges exposure to near-synonymous construction—e.g., she made him laugh) vs. entrenchment (all exposures to a verb’s grammatical usages, including cases like He laughed).

摘要： 学习者如何在没有明确负面证据的情况下避免诸如“Tom laughed me”之类的过度概括？构式语法学家提出了两种描述针对过度概括的间接负面证据的假说：先占（preemption，即优先接触近义构式，例如“she made him laugh”）与固化（entrenchment，即接触动词的所有语法用法，包括“He laughed”等情况）。

We disentangle these hypotheses by running controlled rearing experiments on LMs trained on child-caregiver conversations, where we systematically remove preemptive vs. non-preemptive evidence. We find that while LMs avoid overgeneralizations, they do not show preemption at a verb-specific level, instead showing weak but non-zero evidence of abstract preemption.

我们通过在基于儿童与照料者对话训练的语言模型（LM）上进行受控培养实验，系统地剔除先占性证据与非先占性证据，从而对这些假说进行了拆解。研究发现，虽然语言模型能够避免过度概括，但它们在特定动词层面上并未表现出先占效应，反而表现出微弱但非零的抽象先占证据。

Combined with results from analyzing the LMs’ training dynamics, we find that LMs treat competing structures as indirect positive—as opposed to negative—evidence in the verb-specific condition. Insofar as preemption is the more plausible route to avoiding overgeneralizations in humans, our results point the need for there to be sensitivities to indirect negative evidence in neural network learners, and suggest new human experiments to test abstract preemption.

结合对语言模型训练动态的分析结果，我们发现语言模型在特定动词条件下，将竞争结构视为间接的“正面”证据，而非“负面”证据。鉴于先占机制是人类避免过度概括更合理的途径，我们的研究结果指出，神经网络学习者需要具备对间接负面证据的敏感性，并建议开展新的人类实验来验证抽象先占假说。