ResonatorLM: Causal Resonant Field Mixing for Efficient Long-Context Language Modeling

Abstract: Contemporary language models are dominated by the transformer architecture, which leverages self-attention mechanisms to enable more efficient, parallelized training across a wide set of documents and corpora. This has allowed transformers to effectively model data across a wide range of modalities and contexts.

摘要： 当代语言模型主要由 Transformer 架构主导，该架构利用自注意力机制，能够在广泛的文档和语料库中实现更高效、并行的训练。这使得 Transformer 能够有效地对各种模态和上下文的数据进行建模。

However, transformers, along with their conventional counterparts such as recurrent neural networks (RNNs) and convolutional neural networks (CNNs), often struggle to maintain efficiency when processing long contexts. We introduce ResonatorLM, a new mechanism that replaces attention with a physics-derived alternative.

然而，Transformer 以及循环神经网络 (RNN) 和卷积神经网络 (CNN) 等传统架构，在处理长上下文时往往难以保持效率。我们引入了 ResonatorLM，这是一种用物理衍生替代方案取代注意力机制的新型机制。

ResonatorLM treats token sequences as a single, driven one-dimensional latent field and replaces attention dot products with causal functions of damped resonators. We implement ResonatorLM on a traditional network architecture and test it on standard long-context modeling tasks.

ResonatorLM 将标记序列视为一个单一的、受驱动的一维潜在场，并用阻尼谐振器的因果函数取代了注意力点积。我们在传统网络架构上实现了 ResonatorLM，并在标准的长上下文建模任务中对其进行了测试。

We find that in a small, 6M matched setting, training and prefill speedups increase with sequence length, decode speed reaches 6.47x compared to that of a standard, optimized transformer at 32K tokens, and accuracy reaches 61.31 percent (compared to 55.32 percent) on WikiText.

研究发现，在 6M 参数的小规模匹配设置下，训练和预填充速度随序列长度的增加而提升；在 32K 标记长度下，解码速度达到了标准优化版 Transformer 的 6.47 倍，且在 WikiText 数据集上的准确率达到了 61.31%（相比之下，标准模型为 55.32%）。

---

Paper Details:

• Authors: Archie Chaudhury
• arXiv ID: 2607.05583
• Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
• Submission Date: 6 Jul 2026

论文详情：

• 作者： Archie Chaudhury
• arXiv ID： 2607.05583
• 学科分类： 计算与语言 (cs.CL)；人工智能 (cs.AI)
• 提交日期： 2026 年 7 月 6 日