Stateful Guardrails for Multi-Turn LLM Systems: A Conversational Risk Accumulation Framework

面向多轮大模型系统的状态化护栏：对话风险累积框架

Abstract: Most safety guardrails for large language models (LLMs) evaluate each prompt-response pair in isolation, which misses failures that arise only over a dialogue as benign turns compose into harm. We term this Conversational Risk Accumulation (CRA): gradual intent drift, fragmented assembly of prohibited instructions, and sensitivity build-up from repeated disclosures.

摘要： 目前大多数针对大语言模型（LLM）的安全护栏都是孤立地评估每一对提示词与响应，这导致系统无法识别那些仅在对话过程中、由多个良性轮次组合而成的潜在风险。我们将此现象定义为“对话风险累积”（Conversational Risk Accumulation, CRA）：即意图的逐渐偏移、违规指令的碎片化拼凑，以及因重复披露信息而导致的敏感度累积。

We propose a session-layer CRA Framework that tracks three trajectory signals: semantic drift from a session anchor, a sensitivity-weighted information accumulation graph over extracted entities, and a compliance-gradient signal capturing increasing willingness to comply. For scoring, we provide (i) an unsupervised convex fusion for attribution and ablations, and (ii) CRA-Net DA, a compact learned trajectory model trained with family-adversarial objectives to reduce length and topic-coverage confounds.

我们提出了一个会话层 CRA 框架，用于追踪三种轨迹信号：基于会话锚点的语义偏移、基于提取实体的敏感度加权信息累积图，以及捕捉合规意愿增强的合规梯度信号。在评分方面，我们提供了：(i) 一种用于归因和消融研究的无监督凸融合方法；(ii) CRA-Net DA，这是一种紧凑的轨迹学习模型，通过家族对抗目标进行训练，以减少长度和主题覆盖范围带来的混淆因素。

To benchmark CRA, we release CRA-Bench v0.1 (1,200 eight-turn sessions across three threat families with topic-matched benign twins), CRA-Bench v0.2 (LLM-paraphrased variants to reduce template artifacts), and an extended 5-family set (2,000 sessions adding persona priming and context stuffing). We introduce a trajectory-native evaluation protocol with session-level splits, mixed-set threshold calibration, Trajectory AUROC, turns-to-detection, calibrated false-positive metrics, bootstrap confidence intervals, leave-one-family-out diagnostic stress tests, and synthetic-to-human transfer checks. Claims focus on within-distribution session scoring on CRA-Bench and human-transfer subsets.

为了对 CRA 进行基准测试，我们发布了 CRA-Bench v0.1（包含三个威胁家族的 1,200 个八轮会话，并配有主题匹配的良性对照组）、CRA-Bench v0.2（通过 LLM 改写的变体，以减少模板伪影），以及一个扩展的 5 家族数据集（包含 2,000 个会话，增加了角色引导和上下文填充）。我们引入了一套轨迹原生的评估协议，包括会话级划分、混合集阈值校准、轨迹 AUROC、检测轮次、校准后的误报指标、自助法置信区间、留一家族诊断压力测试以及合成数据到人类数据的迁移检查。研究结论重点关注 CRA-Bench 和人类迁移子集上的分布内会话评分。