Interpreting Latent CoT Reasoning as Dynamical Systems

将潜在思维链（Latent CoT）推理诠释为动力系统

Abstract: Recent latent reasoning methods, such as CODI and COCONUT, face a fundamental interpretability problem: they maintain multiple superimposed candidate traces in the hidden space at each step, unlike explicit-CoT, which follows a single transparent reasoning trace. Existing mechanistic methods show compression, shortcuts, and superposition without explaining how reasoning evolves across latent steps.

摘要： 近期的潜在推理方法（如 CODI 和 COCONUT）面临一个根本性的可解释性问题：与遵循单一透明推理轨迹的显式思维链（explicit-CoT）不同，这些方法在每一步都会在隐藏空间中维持多个叠加的候选轨迹。现有的机制研究方法展示了压缩、捷径和叠加现象，但未能解释推理过程如何在潜在步骤中演化。

To address this gap, we model latent token sequences as trajectories in representation space and apply dynamical systems analysis to characterize the evolution of reasoning. Using quantitative measures, such as step-to-step change, direction consistency, and Lyapunov sensitivity, alongside qualitative projections, such as UMAP and DMD/PHATE, we show that latent CoT exhibits structured, non-random dynamics with two distinct stability classes.

为了填补这一空白，我们将潜在标记序列建模为表征空间中的轨迹，并应用动力系统分析来刻画推理的演化过程。通过使用定量指标（如步间变化、方向一致性和 Lyapunov 敏感度）以及定性投影（如 UMAP 和 DMD/PHATE），我们证明了潜在思维链表现出结构化、非随机的动力学特征，并具有两类截然不同的稳定性。

CODI behaves as a stable attractor, while COCONUT behaves as an unstable expanding system, and SIM-CoT supervision tightens both behaviors without changing the underlying dynamics. This framework advances the interpretability of latent CoT reasoning dynamics and provides actionable insights for improving latent reasoning performance. Code and Project page available online.

CODI 表现为一种稳定的吸引子，而 COCONUT 则表现为一种不稳定的扩张系统；SIM-CoT 监督机制在不改变底层动力学的前提下，强化了这两种行为。该框架提升了对潜在思维链推理动力学的可解释性，并为提高潜在推理性能提供了可操作的见解。代码和项目页面已在线发布。