Nexus: Depth-Adaptive KV-Cache Splicing and Retrieval-Decoupled Tool Routing for Agentic LLMs on Unified Memory

Nexus：面向统一内存架构下智能体大模型的深度自适应 KV 缓存拼接与检索解耦工具路由技术

Agentic large language models (LLMs) on the Model Context Protocol (MCP) re-encode verbose tool schemas every turn, so prefill - quadratic in sequence length - dominates time-to-first-token (TTFT) as the tool registry grows.

在模型上下文协议（MCP）下运行的智能体大模型（LLM）每一轮都需要重新编码冗长的工具模式（Schema），因此随着工具注册库的增长，预填充（Prefill）过程——其计算复杂度与序列长度呈二次方关系——成为了首字延迟（TTFT）的主要瓶颈。

Nexus’s primary lever is to decouple routing from the schema-prefill cost: an INT8 semantic lookaside buffer (SLB) with a calibrated cross-encoder margin gate selects tools by retrieval, and arguments are generated over a compressed textual signature (median 19 tokens) rather than over spliced key/value (KV) cache.

Nexus 的核心手段是将路由与模式预填充成本解耦：通过一个带有校准交叉编码器边缘门控（Margin Gate）的 INT8 语义旁路缓冲区（SLB）进行工具检索选择，并基于压缩后的文本签名（中位数 19 个 Token）而非拼接的键值（KV）缓存来生成参数。

This path is depth-independent: routing accuracy stays near 89% as the registry scales to 250 tools - where a concatenate-all-schemas baseline overflows the context window entirely - and it reaches a first-argument token 1.66x sooner than a full-schema re-prefill at a ~80% main-context token saving.

该路径与深度无关：当注册库扩展至 250 个工具时，路由准确率仍保持在 89% 左右（此时“全模式拼接”基准方案会导致上下文窗口完全溢出），且其首个参数 Token 的生成速度比全模式重新预填充快 1.66 倍，同时节省了约 80% 的主上下文 Token。

As a secondary, bounded lever we transplant a compiled schema KV block directly into the live context. This is fundamentally limited by rotary position embedding (RoPE) phase drift: an anchored splice is output-exact, but off-anchor placement corrupts attention, so beyond a threshold P=256 Nexus repairs the seam with a depth-adaptive suffix redecode that escalates to a full re-prefill.

作为第二种受限手段，我们将编译好的模式 KV 块直接移植到实时上下文中。这在根本上受到旋转位置编码（RoPE）相位漂移的限制：锚定拼接的输出是精确的，但非锚定位置会破坏注意力机制，因此当超过 P=256 的阈值时，Nexus 会通过深度自适应的后缀重解码来修复缝隙，必要时会升级为完整的重新预填充。

The resulting never-regress property is a guarantee on output fidelity (top-1 agreement, D_KL approx. 0) - not on latency, which can dip to 0.98x before converging to parity - alongside a 1.1-1.7x TTFT speedup at moderate depth that narrows to parity at deep context.

由此产生的“永不回退”（Never-regress）特性保证了输出的保真度（Top-1 一致性，D_KL 趋近于 0）——虽然延迟表现并非始终如一（在收敛至持平前可能会降至 0.98 倍），但在中等深度下可实现 1.1-1.7 倍的 TTFT 加速，该优势在深层上下文中会逐渐收窄至持平。

Two negative results bound the design: the off-anchor RoPE fidelity boundary, and the failure of a reference-free drift gate to predict drift (Spearman rho = 0.193). All measurements are from one model tuple (Qwen2.5-14B-Instruct Q4_K_M) on Apple-silicon unified memory; the qualitative boundaries generalize, while the quantitative envelope is tuple-specific.

两项负面结果界定了该设计的边界：非锚定 RoPE 的保真度极限，以及无参考漂移门控在预测漂移方面的失效（Spearman 相关系数 = 0.193）。所有测量数据均基于 Apple Silicon 统一内存上的单一模型元组（Qwen2.5-14B-Instruct Q4_K_M）；其定性边界具有通用性，但定量范围仅适用于该特定模型元组。