A Feasibility Condition for Confidence-Gated Tiered LLM Inference

查看原文 / View Original 编辑 / Edit

Computer Science > Artificial Intelligence arXiv:2611.05394 (cs) [Submitted on 15 Sep 2026] Title: A Feasibility Condition for Confidence-Gated Tiered LLM Inference Authors: Sumit P. Ahuja.

计算机科学 > 人工智能 arXiv:2611.05394 (cs) [提交于 2026 年 9 月 15 日] 标题：置信度门控分层大语言模型推理的可行性条件作者：Sumit P. Ahuja。

Abstract: Tiered inference distributes large language model computation across a hierarchy of devices, from on-device and edge hardware to the datacenter, and is increasingly offered as the route to cheaper, lower-latency, and more private serving. Most such systems orchestrate frozen model checkpoints, and most decide what to resolve locally and what to escalate using a confidence signal produced by a forward-deployed model.

摘要：分层推理将大语言模型的计算分布在从端侧设备、边缘硬件到数据中心的设备层级中，这正日益成为实现更低成本、更低延迟和更高隐私性服务的一种途径。大多数此类系统编排固定的模型检查点，并利用前置部署模型产生的置信度信号来决定哪些任务在本地解决，哪些任务需要升级处理。

We give a closed-form condition for when no threshold on that signal makes a tiered cascade worthwhile relative to sending every query to the backstop. Modeling the routing decision as a binary test and separating local from escalation cost through an architecture cost ratio $\kappa$, we derive a feasibility condition bracketed by a signal-free floor, below which a cascade is feasible at any signal quality, and a perfect-signal ceiling, above which none is.

我们给出了一个闭式条件，用于判断在何种情况下，无论该信号的阈值如何设定，分层级联相比于将所有查询直接发送至后端处理都毫无意义。通过将路由决策建模为二元测试，并利用架构成本比 $\kappa$ 将本地成本与升级成本分离开来，我们推导出了一个可行性条件，该条件由一个“无信号下限”（低于此限，级联在任何信号质量下均可行）和一个“完美信号上限”（高于此限，级联均不可行）所界定。

Read along the $\kappa$ axis the condition inverts: where local compute is billed, cost binds, matching a recent empirical finding; where it is sunk, on decentralized hardware, feasibility becomes unconditional and signal quality instead sets what the cascade achieves. We turn the condition into three design requirements that existing trained-in primitives satisfy only in isolation, and we propose, as future work, a co-designed architecture and a protocol to test it. The condition and the inversion hold independently of that architecture; they are the paper’s contribution.

沿着 $\kappa$ 轴观察，该条件会发生反转：在本地计算需要计费的情况下，成本会产生约束，这与最近的一项实证发现相符；而在去中心化硬件上，当计算成本为沉没成本时，可行性变得无条件，此时信号质量决定了级联所能达到的效果。我们将该条件转化为三个设计要求，而现有的训练原语仅能孤立地满足这些要求；作为未来工作，我们提出了一种协同设计的架构及其测试协议。该条件及其反转特性独立于具体架构，这正是本文的贡献所在。