When Debiasing Backfires: Counterintuitive Side Effects of Preprocessing-Based Stereotype Mitigation

当去偏见适得其反：基于预处理的刻板印象缓解策略的反直觉副作用

Abstract: Preprocessing-based methods for stereotype mitigation, such as pre-/post-training on debiased corpora, are widely used in NLP. While these approaches reduce measurable stereotypes for targeted groups, we find they often induce unintended shifts—side effects, where stereotyping or counter-stereotyping can increase relative to neutral baselines for other demographics, including across unrelated demographic categories.

摘要： 基于预处理的刻板印象缓解方法（例如在去偏语料库上进行预训练或后训练）在自然语言处理（NLP）领域被广泛应用。虽然这些方法减少了针对特定群体的可衡量刻板印象，但我们发现它们往往会引发意想不到的偏移——即副作用，表现为在其他人口统计学群体（包括不相关的人口统计类别）中，刻板印象或反刻板印象相对于中性基准线反而有所增加。

We demonstrate these side effects across two model families (encoder-only and decoder-only), multiple preprocessing strategies (removing stereotypical sentences, removing group mentions, and swapping group references), and both pre- and post-training at different data scales on Wikipedia. Standard benchmarks frequently miss these shifts.

我们在两个模型系列（仅编码器架构和仅解码器架构）、多种预处理策略（删除刻板印象句子、删除群体提及、以及交换群体指代）以及维基百科不同数据规模的预训练和后训练中，均证实了这些副作用的存在。标准的基准测试往往无法捕捉到这些偏移。

Using attention-rollout analysis, we observe that such side effects are not accompanied by large changes in attention flow, complicating mechanistic explanations. We discuss implications for evaluation, provide actionable diagnostics, and argue for side-effect-aware, transparent mitigation practices.

通过注意力展开（attention-rollout）分析，我们观察到这些副作用并不伴随注意力流的显著变化，这使得对其机制的解释变得更加复杂。我们讨论了这些发现对评估工作的影响，提供了可操作的诊断方法，并主张采取具备副作用意识且透明的缓解实践。