Measuring and Improving Behavioral Consistency in Large Language Models through Fact-Heuristic-Emotion State Enforcement

通过事实-启发-情感状态强制机制衡量并提升大语言模型的行为一致性

Abstract: Large language models (LLMs) can give different answers to the same decision problem across runs, and reverse a decision when their own prior answer returns as context. We ask whether this instability can be measured and partially reduced without changing model weights. We test the Cognitive Kernel Model (CKM), a prompt-level state-enforcement layer. Before deciding, the model must separate its input into three epistemic roles: Fact (given or verifiable), Heuristic (inferred or assumed), and Emotion (evaluative or priority signal). CKM adds no capability; it forces the model to track what kind of information it uses before acting. Formally it maintains a structured state S_t = {F_t, H_t, E_t} updated by a transition function.

摘要： 大语言模型（LLM）在面对同一个决策问题时，多次运行可能会给出不同的答案，甚至在将之前的回答作为上下文返回时推翻先前的决定。我们探讨了在不改变模型权重的情况下，是否可以衡量并部分减少这种不稳定性。我们测试了“认知内核模型”（Cognitive Kernel Model, CKM），这是一个提示词层面的状态强制执行层。在做出决策前，模型必须将其输入拆分为三种认知角色：事实（Fact，已知或可验证的）、启发（Heuristic，推断或假设的）以及情感（Emotion，评估或优先级信号）。CKM 不会增加模型的能力，而是强制模型在行动前追踪其所使用的信息类型。在形式上，它维护一个由转换函数更新的结构化状态 S_t = {F_t, H_t, E_t}。

We evaluate CKM on Korean-language decision scenarios (ambiguity, ethical conflict, resource allocation, error handling) across 26 LLMs from four vendors and 37,403 observations, via four core experiments, a 4-arm ablation, a 5-arm sham-restriction ablation, and a temperature probe.

我们通过四项核心实验、一项四臂消融实验、一项五臂伪限制消融实验以及温度探测，在来自四家供应商的 26 个大语言模型上，针对韩语决策场景（包括歧义性、伦理冲突、资源分配、错误处理）进行了 37,403 次观察，以评估 CKM 的效果。

Findings: (1) CKM reduces repeated-output variability (random-effects Hedges’ g=1.09, 95% CI [0.83, 1.35], 31 model pairs); (2) state persistence cuts the decision-flip rate by 82% in newer models (g=1.52); (3) the effect is not JSON formatting alone (value-only recomputation, g=2.24); (4) intrinsic randomness under fixed anchor states is negligible; (5) the advantage grows under sampling stochasticity (g=2.87 at temperature 0.7); (6) a sham ablation attributes about 45% of the gain to structural scaffolding and 55% to Fact/Heuristic/Emotion content, and CKM is the only arm that both raises consistency and reduces flipping. CKM does not improve reasoning correctness. The narrower result: behavioral consistency is measurable, varies across models, and is partially improvable by forcing models to separate facts, assumptions, and evaluative signals before deciding.

研究发现：(1) CKM 降低了重复输出的变异性（随机效应 Hedges’ g=1.09，95% 置信区间 [0.83, 1.35]，涉及 31 对模型）；(2) 在较新的模型中，状态持久性将决策翻转率降低了 82% (g=1.52)；(3) 这种效果并非仅仅源于 JSON 格式化（仅重算数值时 g=2.24）；(4) 在固定锚点状态下，内在随机性可忽略不计；(5) 在采样随机性较高时，该优势更为显著（温度为 0.7 时 g=2.87）；(6) 伪消融实验表明，约 45% 的增益归功于结构化框架，55% 归功于“事实/启发/情感”内容，且 CKM 是唯一能同时提高一致性并减少翻转的方案。CKM 并不能提高推理的正确性。结论总结：行为一致性是可以衡量的，且在不同模型间存在差异；通过强制模型在决策前区分事实、假设和评估信号，可以部分改善这种一致性。