Mapping Text to Multiplex Graph: Prompt Compression as Lévy Walk-Guided Graph Pruning

将文本映射为多路复用图：作为莱维飞行引导图剪枝的提示词压缩

Abstract: Existing prompt compression methods treat text as flat token sequences, failing to capture the distributed nature of important information, which is often spread across multiple locations and connected through both local syntactic dependencies and global semantic relations.

摘要： 现有的提示词压缩方法将文本视为平铺的标记序列，未能捕捉到重要信息分布式的本质。这些信息往往分散在多个位置，并通过局部句法依赖和全局语义关系相互连接。

Such relational structure is naturally represented as a graph, where tokens or sentences become nodes and their dependencies become edges. To this end, we propose RAGP, which formulates prompt compression as Redundancy-Aware Graph Pruning on a multiplex graph that jointly models fine-grained attention-based dependencies and coarse-grained semantic relations.

这种关系结构可以自然地表示为图，其中标记或句子成为节点，而它们的依赖关系成为边。为此，我们提出了 RAGP，它将提示词压缩表述为多路复用图上的“冗余感知图剪枝”（Redundancy-Aware Graph Pruning），该图联合建模了细粒度的基于注意力的依赖关系和粗粒度的语义关系。

To efficiently identify non-redundant nodes in this heterogeneous structure (dense local subgraphs and sparse global connections), we employ Levy walks whose heavy-tailed step distribution naturally balances local exploitation with global exploration.

为了在这种异构结构（密集的局部子图和稀疏的全局连接）中高效识别非冗余节点，我们采用了莱维飞行（Levy walks），其重尾步长分布自然地平衡了局部开发与全局探索。

Experiments on LongBench show that RAGP achieves an average score of 49.3 under a 4x compression ratio, outperforming existing LLM-based compression methods, such as LongLLMLingua, which attains 48.8 at a 3x compression ratio. Besides, RAGP also surpasses state-of-the-art vision-based text compression paradigms on multiple tasks.

在 LongBench 上的实验表明，RAGP 在 4 倍压缩比下取得了 49.3 的平均分，优于现有的基于大语言模型的压缩方法（例如 LongLLMLingua 在 3 倍压缩比下得分为 48.8）。此外，RAGP 在多项任务中也超越了目前最先进的基于视觉的文本压缩范式。