Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts

量化大语言模型推理中的静默失效：基于分类法的“空洞收敛”与失效模式转移分析

Abstract: We show that post-training quantization can silently alter how large language models reason even when task accuracy is preserved. Using a six-category failure taxonomy validated by two independent human annotators (Cohen’s $\kappa$ = 0.906), we classify 30,000 chain-of-thought outputs from five instruction-tuned LLMs (3B—14B parameters) across three quantization precisions (FP32, FP16, NF4) and four reasoning benchmarks.

摘要： 我们研究发现，即使在保持任务准确率的情况下，训练后量化（post-training quantization）也会悄无声息地改变大语言模型的推理方式。我们采用了一套由两名独立人工标注员验证过的六类失效分类法（Cohen’s $\kappa$ = 0.906），对来自五个指令微调大模型（参数量在 3B 到 14B 之间）的 30,000 条思维链（Chain-of-Thought）输出进行了分类，涵盖了三种量化精度（FP32、FP16、NF4）和四个推理基准测试。

We find that while accuracy is robust across precisions (maximum 3.1 pp drop), Hollow Convergence (correct answers reached through incomplete or unverifiable reasoning) shows a significant size-dependent shift under NF4, dropping sharply for the two smallest models tested but remaining invariant for models at 12B parameters and above. This effect is also benchmark-specific: GSM8K is categorically immune while LogiQA and ARC-Challenge show the largest shifts.

研究发现，尽管准确率在不同精度下表现稳健（最大降幅仅为 3.1 个百分点），但在 NF4 量化下，“空洞收敛”（即通过不完整或不可验证的推理得出正确答案）表现出显著的规模依赖性转移：在测试的两个最小模型中，该现象急剧下降，但在 12B 及以上参数的模型中则保持不变。这种效应还具有基准测试特异性：GSM8K 基准测试对此完全免疫，而 LogiQA 和 ARC-Challenge 则表现出最大的偏移。

Furthermore, under NF4, Shortcut Collapse rises from 44% to 78% of wrong-answer failures in LLaMA 3.2-3B while Confidence Snowballing collapses from 15.8% to near zero, a qualitative shift invisible to accuracy metrics. Finally, we show Hollow Convergence cannot be reliably detected from surface-level text features (best F1 = 0.53), establishing it as a deployment-relevant failure mode that standard evaluation pipelines cannot catch.

此外，在 NF4 量化下，LLaMA 3.2-3B 模型中“捷径崩溃”（Shortcut Collapse）在错误答案失效中的占比从 44% 上升至 78%，而“置信度滚雪球”（Confidence Snowballing）则从 15.8% 降至近乎于零——这是一种准确率指标无法察觉的定性转变。最后，我们证明了“空洞收敛”无法通过表层文本特征进行可靠检测（最佳 F1 分数为 0.53），这表明它是一种标准评估流程无法捕捉的、与部署密切相关的失效模式。