Latent Space Refusal Anchoring for Low-Resource African Languages: Mechanistic Safety Recovery Without Retraining

低资源非洲语言的潜在空间拒绝锚定：无需重训练的机制化安全恢复

Abstract: Instruction-tuned models often refuse harmful requests in English but comply with the same requests in Yoruba, Igbo, Igala, and Hausa. This suggests that the refusal mechanism is present in the residual stream but fails to activate for low-resource inputs. Recovering it normally requires labelled target-language data and retraining, neither of which is available at scale for most African languages.

摘要： 指令微调模型通常会拒绝英语中的有害请求，但在处理约鲁巴语（Yoruba）、伊博语（Igbo）、伊加拉语（Igala）和豪萨语（Hausa）的相同请求时却会予以配合。这表明拒绝机制存在于残差流（residual stream）中，但无法针对低资源语言输入被激活。通常情况下，恢复该机制需要标注的目标语言数据和重训练，而大多数非洲语言在大规模应用中均缺乏这些资源。

We introduce Latent Space Refusal Anchoring (LSR-Anchoring), a training-free method that extracts the refusal direction from English prompts and clamps it onto the residual stream at inference time. The primary variant, Mean-Activation Steering (MAS), operates across the four architectures we tested: Llama-3-8B, Llama-3.1-70B, Mistral-7B-Instruct, and Qwen2.5-7B. On Mistral and Qwen it recovers safety with benign degradation below 0.08. On Llama-3-8B it overcorrects, with Degraded Performance on Legitimate prompts (DPL) reaching 1.00.

我们引入了潜在空间拒绝锚定（LSR-Anchoring），这是一种无需训练的方法，通过从英语提示中提取拒绝方向，并在推理时将其固定（clamp）到残差流上。其主要变体“平均激活引导”（Mean-Activation Steering, MAS）在我们测试的四种架构上均有效：Llama-3-8B、Llama-3.1-70B、Mistral-7B-Instruct 和 Qwen2.5-7B。在 Mistral 和 Qwen 上，它在恢复安全性的同时，良性性能下降（benign degradation）低于 0.08。在 Llama-3-8B 上，它出现了过度修正，导致合法提示的性能下降（DPL）达到了 1.00。

We address this with SAE-Derived Steering (SDS), which replaces the dense mean-difference direction with a single Sparse Autoencoder (SAE) feature and reduces Kullback-Leibler (KL) divergence by 3.5-7x without benign collapse. Four languages transfer positively, but Arabic fails on every architecture and at every steering magnitude, indicating a geometric mismatch rather than a baseline effect. Massive Multitask Language Understanding (MMLU) accuracy drops remain below 0.35 percentage points at every effective steering magnitude.

我们通过 SAE 衍生引导（SAE-Derived Steering, SDS）解决了这一问题，该方法用单个稀疏自编码器（SAE）特征替换了密集的平均差方向，并将 Kullback-Leibler (KL) 散度降低了 3.5 到 7 倍，且未发生良性崩溃。四种语言均表现出正向迁移，但阿拉伯语在所有架构和所有引导强度下均失败，这表明是几何不匹配而非基线效应所致。在所有有效的引导强度下，大规模多任务语言理解（MMLU）的准确率下降均保持在 0.35 个百分点以下。