Inference-Time Mitigation of Adversarial Political Bias in Large Language Models

大语言模型中对抗性政治偏见的推理时缓解策略

Abstract: As Large Language Models (LLMs) become the mainstay for information retrieval and summarization tasks, ensuring that they are always non-partisan and invulnerable to political bias is a critical step towards safer and more trustworthy Artificial Intelligence (AI).

摘要： 随着大语言模型（LLMs）成为信息检索和摘要任务的主流工具，确保其始终保持中立且不受政治偏见影响，是迈向更安全、更可信人工智能（AI）的关键一步。

Current model alignment paradigms, such as reinforcement learning from human feedback (RLHF), make LLMs follow overarching safety instructions. However, this instruction tuning can be exploited via adversarial prompt injection and be used to generate unsafe content. In particular, political bias has not been specifically targeted by modern alignment techniques as harmful and biased content.

当前的模型对齐范式（如基于人类反馈的强化学习，即 RLHF）旨在使大语言模型遵循总体安全指令。然而，这种指令微调可能会被对抗性提示注入所利用，进而生成不安全的内容。特别是，现代对齐技术尚未将政治偏见作为有害和偏见内容进行专门处理。

To address this vulnerability of LLMs, we propose mitigation strategies using Chain of Thought (CoT) prompting and Direct Preference Optimization (DPO). Using a public dataset of legislative videos, we generate summaries using LLMs, inject bias via adversarial prompting and evaluate their performance on a four axis scale designed for political summarization.

为了解决大语言模型的这一脆弱性，我们提出了利用思维链（CoT）提示和直接偏好优化（DPO）的缓解策略。我们使用一个公开的立法视频数据集，通过大语言模型生成摘要，利用对抗性提示注入偏见，并根据为政治摘要设计的四轴量表评估其性能。

In this paper, we present different methods to shield LLMs against the injection of political bias. Our results demonstrate that the proposed Recursive Self-Correction approach raises model performance from a Political Neutrality Likert scale baseline of 2.14 to 4.56, averaged across all models, demonstrating effective inference-time mitigation of political bias in LLM-generated summaries.

在本文中，我们提出了多种保护大语言模型免受政治偏见注入的方法。研究结果表明，所提出的递归自我修正（Recursive Self-Correction）方法将模型在政治中立性李克特量表上的表现从基准的 2.14 提升至 4.56（所有模型的平均值），证明了在推理阶段有效缓解大语言模型生成摘要中政治偏见的可行性。