Neural Bayesian Sequential Routing

Abstract: Human decision-making is sequential and uncertainty-aware, yet standard neural networks often rely on static, dense forward computation with limited visibility into evidence acquisition, uncertainty evolution, or when computation should stop. We introduce Neural Bayesian Sequential Routing (NBSR), a framework that models neural inference as active evidence accumulation over a hierarchical Directed Acyclic Graph (DAG).

摘要： 人类的决策过程是序列化且具备不确定性意识的，然而标准的神经网络往往依赖于静态、密集的正向计算，且在证据获取、不确定性演变或何时停止计算等方面缺乏透明度。我们引入了“神经贝叶斯序列路由”（Neural Bayesian Sequential Routing, NBSR），这是一个将神经推理建模为在分层有向无环图（DAG）上进行主动证据积累的框架。

Within a Dirichlet—Categorical conjugate framework, neural experts query a persistent global knowledge oracle to extract positive evidence vectors, which act as pseudo-counts and update a Dirichlet belief state by exact conjugate addition. Coupled with a Gumbel-Softmax Straight-Through estimator, this update enables hard, path-dependent routing while preserving surrogate gradients for end-to-end training.

在狄利克雷-分类（Dirichlet-Categorical）共轭框架内，神经专家通过查询持久的全局知识预言机来提取正向证据向量；这些向量充当伪计数，并通过精确的共轭加法更新狄利克雷信念状态。结合 Gumbel-Softmax 直通估计器（Straight-Through estimator），这种更新实现了硬性、路径依赖的路由，同时保留了用于端到端训练的代理梯度。

The resulting Dirichlet precision and entropy provide mechanisms for uncertainty quantification, entropy-based early exiting, OOD abstention, and cost-aware evidence acquisition. We prove that, under strictly positive evidence extraction, total Dirichlet precision increases monotonically along any valid trajectory and marginal predictive variance is bounded, formalizing sequential “hypothesis sharpening”; under idealized capacity and optimization assumptions, the terminal Dirichlet expectation recovers the Bayes-optimal conditional distribution.

由此产生的狄利克雷精度和熵为不确定性量化、基于熵的提前退出、分布外（OOD）弃权以及成本敏感的证据获取提供了机制。我们证明，在严格正向证据提取的条件下，狄利克雷总精度沿任何有效轨迹单调增加，且边际预测方差是有界的，从而形式化了序列化的“假设锐化”（hypothesis sharpening）；在理想化的容量和优化假设下，最终的狄利克雷期望可以恢复贝叶斯最优条件分布。

Empirical evaluations across visual categorization, structured medical diagnosis, language modeling, partially observable control, and cost-aware Bayesian experimental design show that NBSR achieves competitive predictive performance while providing transparent routing traces, path-dependent evidence attribution, uncertainty-aware decision control, and resource-rational inference. Overall, NBSR offers a mathematically grounded framework for interpretable, modular, and resource-rational agentic AI.

在视觉分类、结构化医学诊断、语言建模、部分可观测控制以及成本敏感的贝叶斯实验设计等方面的实证评估表明，NBSR 在实现具有竞争力的预测性能的同时，还提供了透明的路由轨迹、路径依赖的证据归因、不确定性感知决策控制以及资源理性的推理。总而言之，NBSR 为可解释、模块化和资源理性的智能体 AI 提供了一个数学基础扎实的框架。