DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions

DecodeShare：追踪大语言模型解码阶段决策的共享子空间

Large language models (LLMs) handle many tasks with one set of parameters, but under KV-cached inference it is unclear what task-general structure, if any, is used at decode time rather than during prefill. 大语言模型（LLMs）使用同一套参数处理多种任务，但在 KV 缓存推理模式下，尚不清楚在解码阶段（decode time）相比预填充阶段（prefill）是否使用了某种任务通用的结构。

We propose DecodeShare, a protocol that identifies a low-dimensional subspace consistently shared across tasks in decode-time hidden states, and then tests its causal role by removing that subspace only during decoding. 我们提出了 DecodeShare，这是一种能够识别解码阶段隐藏状态中跨任务共享的低维子空间的协议，并通过仅在解码过程中移除该子空间来测试其因果作用。

In our experiments, disturbing the discovered shared subspace degrades decision performance far more than disturbing either a prefill-derived or random subspace under the same intervention budget. 实验表明，在相同的干预预算下，干扰所发现的共享子空间对决策性能的损害，远大于干扰预填充阶段导出的子空间或随机子空间。

We further show this decode-shared subspace has practical consequences for activation steering: common steering directions can overlap the task-general decode channel. 我们进一步证明，这种解码共享子空间对激活引导（activation steering）具有实际意义：常见的引导方向可能与任务通用的解码通道重叠。

Projecting out this shared subspace directly separates the functional roles of the two components, while evaluating steering vectors at decode-time yields more reliable signal for downstream deployment than prefill-based proxies. 通过投影移除该共享子空间，可以直接分离这两个组件的功能角色；同时，在解码阶段评估引导向量，比基于预填充的代理方法能为下游部署提供更可靠的信号。

Despite its compactness, the shared subspace can serve as a high-leverage causal channel at decode time. Code is available at: this https URL. 尽管该共享子空间结构紧凑，但它可以在解码阶段充当高杠杆的因果通道。代码地址：点击此处。