Prediction Is Not Calibration: Item Parameter Recovery and Representation Coupling in Knowledge Tracing
Knowledge tracing (KT) models can estimate item parameters through an item response decoder trained solely for next-response prediction, yet predictive accuracy does not guarantee parameter recovery. We study 2PL, GPCM, and NRM decoders paired with LSTM, Transformer, and DKVMN encoders.
知识追踪（KT）模型可以通过仅针对下一次响应预测进行训练的项目响应解码器来估计项目参数，但预测准确性并不能保证参数的恢复。我们研究了与 LSTM、Transformer 和 DKVMN 编码器配对的 2PL、GPCM 和 NRM 解码器。
Response-model derivatives show why discrimination and option slope updates depend directly on the scale and spread of estimated ability. Affine logit identities show how item-dependent shifts and rescaling in estimated ability are absorbed into item parameters without changing response probabilities.
响应模型导数显示了为什么区分度和选项斜率的更新直接取决于估计能力的规模和分布。仿射 Logit 恒等式展示了估计能力中与项目相关的偏移和重缩放是如何在不改变响应概率的情况下被吸收到项目参数中的。
In simulations, difficulty, mean threshold, and correct-option intercept recovery remain high under shuffled order across representation widths, whereas discrimination and keyed option slope contrast recovery require wider item representations and degrade further under fixed item order.
在模拟中，无论表示宽度如何，在顺序打乱的情况下，难度、平均阈值和正确选项截距的恢复率仍然很高，而区分度和关键选项斜率对比的恢复则需要更宽的项目表示，并且在固定项目顺序下会进一步下降。
Separating the sequence item representation (width $d_S$) from the item parameter key (width $d_M$), and pairing a wide key with a thin sequence representation ($\mathrm{SK}(64,8)$), reduces fixed-order ability distortions and improves recovery and cross-order consistency without sacrificing test accuracy or increasing model size relative to a shared width-64 baseline ($\mathrm{SH}(64)$).
将序列项目表示（宽度 $d_S$）与项目参数键（宽度 $d_M$）分离，并将宽键与窄序列表示配对（$\mathrm{SK}(64,8)$），可以在不牺牲测试准确性或增加模型大小的情况下（相对于共享宽度为 64 的基线 $\mathrm{SH}(64)$），减少固定顺序的能力失真，并提高恢复效果和跨顺序的一致性。
On TIMSS and EdNet, widening $d_M$ consistently improves agreement with an offline IRT reference, while the trade-off from narrowing $d_S$ depends on the encoder’s reliance on sequence width for prediction. These findings show why item parameters, especially discrimination and option slopes, must be evaluated separately from response prediction.
在 TIMSS 和 EdNet 数据集上，加宽 $d_M$ 始终能提高与离线 IRT 参考的一致性，而缩小 $d_S$ 带来的权衡则取决于编码器对序列宽度进行预测的依赖程度。这些发现表明，为什么项目参数（尤其是区分度和选项斜率）必须与响应预测分开进行评估。