LLMs Silently Correct African American English: Auditing and Mitigating Dialect Bias via Activation Steering
LLMs Silently Correct African American English: Auditing and Mitigating Dialect Bias via Activation Steering
大语言模型对非裔美国人英语的隐性修正：通过激活引导审计并缓解方言偏见
Abstract: African American English (AAE), a rule-governed dialect spoken by over 30 million people, is routinely misinterpreted and “corrected” by large language models (LLMs). Across six instruction-tuned LLMs (14B to 70B), we show that state-of-the-art models systematically prefer Standard American English (SAE) continuations even when the preceding context is in AAE, effectively rewriting AAE into SAE.
摘要： 非裔美国人英语（AAE）是一种拥有超过 3000 万使用者的规则化方言，却经常被大语言模型（LLM）误解并进行“修正”。通过对六个经过指令微调的大语言模型（参数量从 14B 到 70B 不等）进行测试，我们发现，即使前文语境为 AAE，这些顶尖模型在生成后续内容时仍系统性地偏向标准美式英语（SAE），从而有效地将 AAE 重写为 SAE。
We present an end-to-end framework to audit and mitigate this bias. For auditing, we introduce conditional Dialect Group Invariance (cDGI), which isolates true model bias from translator-induced artifacts, and a feature-level localization analysis that identifies which AAE markers most strongly trigger bias; we find that syntactic constructions, especially negative concord (e.g., “ain’t nobody”), are universal triggers across all models.
我们提出了一个端到端的框架来审计并缓解这种偏见。在审计方面，我们引入了条件方言组不变性（cDGI），它能将模型真正的偏见与翻译过程中产生的伪影区分开来；同时，我们通过特征级定位分析，识别出哪些 AAE 标记最容易触发偏见。研究发现，句法结构，尤其是否定一致（例如“ain’t nobody”），是所有模型中普遍存在的触发因素。
For mitigation, we introduce, to our knowledge, the first application of activation steering to dialect bias: a training-free, test-time method that extracts dialect directions via causal tracing and injects them into bias-relevant layers. Activation steering reduces bias 5 to 20 times more than prompting while preserving SAE fluency.
在缓解偏见方面，据我们所知，我们首次将激活引导（activation steering）应用于方言偏见问题：这是一种无需训练、在测试阶段使用的方法，通过因果追踪提取方言方向，并将其注入到与偏见相关的层中。与提示工程（prompting）相比，激活引导在保持 SAE 流畅度的同时，能将偏见降低 5 到 20 倍。
To enable this work, we release REAL-AAE, the largest real-AAE parallel corpus to date: 17,479 AAE/SAE/AAE_back triplets from natural tweets (2 to 6 times larger than prior real-AAE resources), validated automatically (BERTScore F1 = 0.95) and by three native AAE speakers (83.0% semantic agreement).
为了支持这项研究，我们发布了 REAL-AAE，这是迄今为止最大的真实 AAE 平行语料库：包含 17,479 个来自真实推文的 AAE/SAE/AAE_back 三元组（比以往的真实 AAE 资源大 2 到 6 倍），并经过了自动验证（BERTScore F1 = 0.95）以及三位 AAE 母语使用者的验证（语义一致性达 83.0%）。