Lexical Prompt Compression for Large Language Models: A Training-Free, Deterministic Pipeline with Empirical Pareto Analysis Across Eleven Task Categories
Lexical Prompt Compression for Large Language Models: A Training-Free, Deterministic Pipeline with Empirical Pareto Analysis Across Eleven Task Categories
大型语言模型的词法提示压缩：一种无需训练、确定性的流水线，涵盖十一个任务类别的经验帕累托分析
Abstract: Recent advances in large language models (LLMs) have made prompts increasingly large and complex. Techniques such as chain-of-thought reasoning (Wei et al., 2022) and in-context learning (Brown et al., 2020) frequently push real-world prompts past several thousand tokens, increasing inference cost and latency.
摘要： 大型语言模型（LLM）的最新进展使得提示词（prompts）变得日益庞大且复杂。诸如思维链推理（Wei et al., 2022）和上下文学习（Brown et al., 2020）等技术经常将实际应用中的提示词推向数千个 token 以上，从而增加了推理成本和延迟。
Learned compression methods such as LLMLingua (Jiang et al., 2023) and Selective Context (Li et al., 2023) achieve high compression ratios but require auxiliary language models and are non-deterministic. We ask a complementary question: how far can a training-free, fully deterministic, CPU-only pipeline based on classical lexical NLP be pushed before output quality degrades significantly?
诸如 LLMLingua (Jiang et al., 2023) 和 Selective Context (Li et al., 2023) 等学习型压缩方法虽然实现了高压缩比，但需要辅助语言模型，且具有非确定性。我们提出了一个互补的问题：基于经典词法自然语言处理（NLP）、无需训练、完全确定性且仅在 CPU 上运行的流水线，在输出质量显著下降之前，究竟能达到多大的压缩极限？
Eleven toggleable lexical transformations - stopword removal, filler-phrase deletion, contraction and abbreviation substitution, part-of-speech-based pruning, lemmatization, WordNet-driven synonym shortening, and named-entity preservation - are assembled into a configurable pipeline.
我们构建了一个可配置的流水线，集成了十一种可切换的词法转换技术，包括：停用词移除、填充短语删除、缩写与简写替换、基于词性的剪枝、词形还原、基于 WordNet 的同义词缩短以及命名实体保留。
Fifteen configurations are evaluated on 1,242 English-only prompts from six sources (Dolly-15k, LMSYS-Chat-1M, WildChat-1M, MMLU, GSM8K, HellaSwag), spanning eleven automatically derived task categories, yielding 18,630 paired GPT-4o-mini completions. Output preservation is measured using BLEU, ROUGE-1/2/L, BERTScore-F1, and SentenceBERT cosine similarity.
我们对来自六个来源（Dolly-15k, LMSYS-Chat-1M, WildChat-1M, MMLU, GSM8K, HellaSwag）的 1,242 条纯英文提示词进行了十五种配置的评估，涵盖了十一个自动派生的任务类别，共产生了 18,630 对 GPT-4o-mini 的补全结果。输出的保留质量通过 BLEU、ROUGE-1/2/L、BERTScore-F1 和 SentenceBERT 余弦相似度进行衡量。
The most aggressive configuration achieves a mean token reduction of 40.3% (sigma = 9.2) at a BERTScore-F1 of 0.876 against the original-prompt output; a stopword-only configuration achieves 29.6% reduction at 0.913. The compression-versus-fidelity Pareto frontier is characterized per task category, with commonsense reasoning a systematic failure mode under aggressive compression. All code, prompts, and per-cell results are released for reproducibility.
最激进的配置实现了 40.3%（标准差 = 9.2）的平均 token 缩减率，其 BERTScore-F1 为 0.876（对比原始提示词输出）；仅使用停用词移除的配置则实现了 29.6% 的缩减率，BERTScore-F1 为 0.913。我们针对每个任务类别刻画了压缩率与保真度之间的帕累托前沿（Pareto frontier），发现常识推理在激进压缩下表现出系统性的失效模式。所有代码、提示词及各单元格结果均已公开，以供复现。