LayerRAG-Bench: A Cross-Layer Reliability Benchmark for Agentic Retrieval-Augmented Generation

LayerRAG-Bench：面向智能体检索增强生成（RAG）的跨层可靠性基准测试

Abstract: Agentic retrieval-augmented generation systems can produce answers that appear grounded while failing at the evidence, tool-contract, authorization, or session-state layer. 摘要： 智能体检索增强生成（Agentic RAG）系统可能会生成看似有据可查的答案，但实际上却在证据层、工具契约层、授权层或会话状态层出现故障。

We introduce LayerRAG-Bench, a controlled cross-layer reliability benchmark with 8 enterprise domains, 240 tasks, 9 fault scenarios, 2 contract modes, and 38,880 live task-level records across nine models from OpenAI, Anthropic, and Gemini. 我们推出了 LayerRAG-Bench，这是一个受控的跨层可靠性基准测试，涵盖了 8 个企业领域、240 个任务、9 种故障场景、2 种契约模式，以及来自 OpenAI、Anthropic 和 Gemini 九种模型的 38,880 条实时任务级记录。

Schema normalization raises schema-drift success from 0.000 to 0.913, but stale evidence, missing tool output, denied permissions, and wrong-session context are not recovered by schema normalization. 模式标准化（Schema normalization）将模式漂移（schema-drift）的成功率从 0.000 提升至 0.913，但对于陈旧证据、缺失的工具输出、拒绝访问权限以及错误的会话上下文，模式标准化无法进行修复。

Groundedness-only evaluation also produces substantial false positives under stale and wrong-session evidence. 仅基于“有据可查性”（Groundedness-only）的评估在面对陈旧证据和错误会话证据时，也会产生大量的误报。

These results support a layer-specific evaluation principle: a reliability intervention should be credited for repairing its target layer without being mistaken for a universal fix. 这些结果支持了一项分层评估原则：可靠性干预措施应因修复了其目标层而受到认可，而不应被误认为是通用的解决方案。