Safe Error Correction for Language Models: Frozen-Base Adjustment with Capability Preservation

语言模型安全纠错：在保持基础能力的前提下进行冻结基座调整

Abstract: We study a practical question: can a small correction module fix errors in a frozen language model’s outputs without degrading its base capabilities? We propose CRN v2, a lightweight logit-level correction module (~34M trainable parameters, 0.73% of the 4.65B text module) that sits atop a fully frozen Gemma 4 E2B model. The base model is never updated; only the correction module learns, via supervised fine-tuning followed by reference-free DPO on 83,400 error-correction pairs.

摘要： 我们研究了一个实际问题：一个小型纠错模块能否在不降低语言模型基础能力的前提下，修复其输出中的错误？我们提出了 CRN v2，这是一个轻量级的 Logit 级纠错模块（约 3400 万个可训练参数，占 46.5 亿参数文本模型规模的 0.73%），它被置于完全冻结的 Gemma 4 E2B 模型之上。基座模型不会进行任何更新；只有纠错模块通过监督微调（SFT）以及随后的无参考直接偏好优化（DPO），在 83,400 个纠错对上进行学习。

On a 60-question domain exam (CEHRI: Certified Human-Robot Intelligence, covering facts, arithmetic, and implicit-goal reasoning), CRN v2 corrects 53.3% of base-model errors (reworded variant: 43.3%) while showing no degradation on tested capability benchmarks (MMLU/BoolQ N=200; car-wash N=8). A LoRA baseline at the matched CRN v1 budget (6.6M params, rank 19) achieves 83.3% correction but suffers 30-75% capability loss on the same benchmarks — the correction-capability tradeoff.

在一项包含 60 道题的领域考试（CEHRI：认证人机智能，涵盖事实、算术和隐式目标推理）中，CRN v2 修复了基座模型 53.3% 的错误（改写变体为 43.3%），同时在测试的能力基准（MMLU/BoolQ N=200；car-wash N=8）上未表现出任何性能下降。相比之下，在匹配 CRN v1 预算（660 万参数，秩 19）下的 LoRA 基线模型虽然达到了 83.3% 的纠错率，但在相同基准测试中却遭受了 30-75% 的能力损失——这体现了纠错与能力之间的权衡。

An ablation shows that the KL preservation term (lambda=0.1) is critical: lowering it to 0.01 degrades correction to 35.0%. A hidden-state injection variant at earlier layers (1.6M params, SFT-only) reaches 50.0%/55.8% but does not exceed logit correction; shallower injection (layer 4) drops to 30.0%/28.3%; multi-depth logit correction (~35M) reaches only 40%; and longer training (5,000 SFT + 2,000 DPO) stays at 53.3% — none of the alternative configurations we tested exceeded the rank-128 logit result, consistent with a best-achieved result of ~53% rather than a floor.

消融实验表明，KL 保持项（lambda=0.1）至关重要：将其降低至 0.01 会使纠错率下降至 35.0%。在早期层进行隐藏状态注入的变体（1.6M 参数，仅 SFT）达到了 50.0%/55.8%，但并未超过 Logit 纠错的效果；更浅层的注入（第 4 层）下降至 30.0%/28.3%；多深度 Logit 纠错（约 35M）仅达到 40%；更长时间的训练（5,000 SFT + 2,000 DPO）仍维持在 53.3%——我们测试的所有替代配置均未超过秩 128 的 Logit 结果，这与约 53% 的最佳实现结果一致，而非一个基准下限。

This is a study of a design principle (frozen base + logit correction + KL anchoring), not a claim of architectural novelty. All code, main-result weights, and evaluation scripts are released (deep variant as code only — no trained deep checkpoints).

这是一项关于设计原则（冻结基座 + Logit 纠错 + KL 锚定）的研究，而非对架构创新的宣称。所有代码、主要结果权重和评估脚本均已发布（深度变体仅提供代码，不提供训练好的深度检查点）。