RENDER: Controlling Reader-Facing Evidence in LLM Memory Evaluation

Title: RENDER: Controlling Reader-Facing Evidence in LLM Memory Evaluation 标题： RENDER：在大语言模型记忆评估中控制面向读者的证据

Abstract: Memory and RAG evaluations often treat the answering model’s input as an implementation detail, even though systems may render the same history as a memory entry, summary, typed record, or raw excerpt. We introduce RENDER, a benchmark control that fixes the conversation while varying the reader-facing artifact. 摘要： 记忆和检索增强生成（RAG）评估通常将回答模型的输入视为一种实现细节，尽管系统可能会将相同的历史记录呈现为记忆条目、摘要、类型化记录或原始摘录。我们引入了 RENDER，这是一种基准控制方法，它在固定对话内容的同时，改变面向读者的呈现形式（artifact）。

RENDER combines a five-level packet ladder, localizing when answer-bearing content enters the input, with deterministic templates approximating ChatGPT-style entries, LangChain summaries, MemGPT-style typed records, and raw conversation. RENDER 结合了一个五级数据包阶梯（packet ladder），用于定位包含答案的内容何时进入输入端，并配合确定性模板，模拟了 ChatGPT 风格的条目、LangChain 摘要、MemGPT 风格的类型化记录以及原始对话。

On 500 LongMemEval questions and nine models, matched-budget resolved packets beat recency-truncated raw dialogue by 42.4-72.6 points. In deployed-style templates, best-worst spread is 24.6-48.8 points per model; under the primary scorer, ChatGPT-style entries have higher point estimates than raw conversation on 7 of 9 models. 在 500 个 LongMemEval 问题和九个模型上的测试显示，匹配预算后的解析数据包比按近因截断的原始对话高出 42.4 到 72.6 分。在部署风格的模板中，每个模型的最好与最差表现差距为 24.6 到 48.8 分；在主要评分标准下，9 个模型中有 7 个模型显示 ChatGPT 风格的条目比原始对话具有更高的得分估计。

Judge rescoring preserves the positive aggregate effect, but model-specific significance is mixed. Three models scoring 0 percent on formal ledger packets answer the same facts from natural-language entries at 45.4-53.4 percent. 裁判模型重新评分后，整体的正向效应依然存在，但针对特定模型的显著性表现不一。三个在正式账本数据包上得分为 0% 的模型，在处理自然语言条目时，对相同事实的回答准确率达到了 45.4% 到 53.4%。

The effect persists under retrieval noise and transfers to HotpotQA, suggesting that memory/RAG evaluations should report or control the reader-facing artifact. 这种效应在检索噪声下依然存在，并可迁移至 HotpotQA，这表明记忆/RAG 评估应当报告或控制面向读者的呈现形式。