TRACE: Transformer Residual Activation Codebook Encoding for Communication-efficient Distributed Edge Inference

查看原文 / View Original 编辑 / Edit

Computer Science > Machine Learning arXiv:2610.22435 (cs) [Submitted on 15 Oct 2026] Title: TRACE: Transformer Residual Activation Codebook Encoding for Communication-efficient Distributed Edge Inference Authors: Muhammad Azlan Qazi, Alexandros Iosifidis, Qi Zhang.

计算机科学 > 机器学习 arXiv:2610.22435 (cs) [提交于 2026 年 10 月 15 日] 标题：TRACE：用于通信高效分布式边缘推理的 Transformer 残差激活码本编码作者：Muhammad Azlan Qazi, Alexandros Iosifidis, Qi Zhang。

Abstract: Distributed Transformer inference across co-located edge devices is fundamentally communication-bound, as exchanging activations between sequence-parallel partitions dominates inference latency under commodity edge bandwidths. Existing compression schemes reduce this cost but retain a structural bottleneck: the received data must still undergo key/value projection before attention can proceed, serializing computation behind communication.

摘要：在共址边缘设备上进行的分布式 Transformer 推理从根本上受限于通信，因为在商品化边缘带宽下，序列并行分区之间交换激活值主导了推理延迟。现有的压缩方案降低了这一成本，但仍保留了一个结构性瓶颈：接收到的数据在注意力机制进行之前必须经过键/值投影，从而导致计算必须在通信之后串行执行。

This paper presents Transformer Residual Activation Codebook Encoding (Trace), a framework that addresses these limitations. Trace places a Vector Quantizer (VQ) at the post-attention residual of every Transformer layer and precomputes codebook-indexed key/value representations. During inference, exchanged activations are encoded into compact codebook indices, allowing receivers to directly recover keys and values through table lookup instead of executing the downstream computations required to derive them.

本文提出了 Transformer 残差激活码本编码（Trace），这是一个解决上述局限性的框架。Trace 在每个 Transformer 层的注意力残差之后放置了一个向量量化器（VQ），并预先计算了码本索引的键/值表示。在推理过程中，交换的激活值被编码为紧凑的码本索引，允许接收方通过查表直接恢复键和值，而无需执行推导它们所需的下游计算。

By eliminating the received data downstream computation, Trace enables communication—computation overlap and reduces a key latency bottleneck in distributed Transformer inference. A hierarchical clustering further structures the learned codebook into a multi-resolution hierarchy, enabling runtime adaptation of the communication bitrate without retraining.

通过消除接收数据的下游计算，Trace 实现了通信与计算的重叠，并减少了分布式 Transformer 推理中的一个关键延迟瓶颈。分层聚类进一步将学习到的码本结构化为多分辨率层次结构，从而无需重新训练即可实现通信比特率的运行时自适应。

The framework supports two operating variants: one prioritizing accuracy and the other balancing accuracy and efficiency through buffer-compatible quantization. Experiments on ViT-Base (ImageNet-1K and CIFAR-100) and ViViT-B (UCF-101) show that Trace achieves up to 1.89% higher accuracy and up to 1.27x end-to-end inference speedup compared with the state-of-the-art VQ-based distributed inference framework. Compared with uncompressed sequence-parallel inference, Trace reduces communication overhead by up to 22,624x.

该框架支持两种运行变体：一种优先考虑准确性，另一种通过缓冲区兼容的量化来平衡准确性和效率。在 ViT-Base（ImageNet-1K 和 CIFAR-100）和 ViViT-B（UCF-101）上的实验表明，与最先进的基于 VQ 的分布式推理框架相比，Trace 的准确率提高了 1.89%，端到端推理速度提升了 1.27 倍。与未压缩的序列并行推理相比，Trace 将通信开销降低了高达 22,624 倍。