I Built Non-Autoregressive Decision Models a Year Ago. Then a Frontier Lab Called It a “Breakthrough”

我在一年前就构建了非自回归决策模型，随后一家前沿实验室将其称为“突破”

From our March 2025 arXiv paper on RL conversion trajectories to building a sub-40ms open-weight System 1 decision engine with RLCD. Everyone in AI right now is talking about a new kind of model: an architecture that is not auto-regressive, does not generate text, and gives lightning fast probability predictions over a structured JSON schema. 从我们 2025 年 3 月关于强化学习（RL）转化轨迹的 arXiv 论文，到构建基于 RLCD 的 40 毫秒内响应的开源权重“系统 1”（System 1）决策引擎。目前 AI 领域每个人都在谈论一种新型模型：一种非自回归、不生成文本，且能针对结构化 JSON 模式提供闪电般快速概率预测的架构。

Seeing the hype online feels both validating and deeply frustrating. I worked on this literally one year back in March 2025. I spent months of hard work, sweat, and sleepless nights building it, published an arXiv paper (arXiv:2503.23303), released the model weights on Hugging Face (sales-conversion-model-reinf-learning), published the open dataset (saas-sales-conversations), built a PyPi package, and posted the whole approach on Reddit (r/LocalLLaMA post). 看到网上的炒作，我既感到被认可，又感到深深的沮丧。我确实在 2025 年 3 月就着手研究这个课题了。我花费了数月的心血、汗水和不眠之夜来构建它，发表了 arXiv 论文（arXiv:2503.23303），在 Hugging Face 上发布了模型权重（sales-conversion-model-reinf-learning），发布了开放数据集（saas-sales-conversations），构建了 PyPi 包，并将整个方法发布在了 Reddit 上（r/LocalLLaMA 帖子）。

Then in September 2025, I published a second paper (arXiv:2510.01237), laying out the exact framework for schema-based decisions guided by reinforcement learning. For anyone curious, the guiding brain in my system was always reinforcement learning, not just an embedding model or an autoregressive LLM. 随后在 2025 年 9 月，我发表了第二篇论文（arXiv:2510.01237），详细阐述了由强化学习引导的基于模式（schema-based）决策的完整框架。对于好奇的人，我的系统中的核心大脑始终是强化学习，而不仅仅是一个嵌入模型或自回归大语言模型（LLM）。

And then in September 2026, a well-funded frontier lab called TypeSafe AI (founded by Diogo Almeida, a co-inventor of ChatGPT at OpenAI) launched Jev. They proposed the exact same non-autoregressive decision concept as if it was a brand-new scientific breakthrough. Except they launched without technical papers, without open weights, and with zero open training datasets. 接着在 2026 年 9 月，一家资金雄厚的前沿实验室 TypeSafe AI（由 OpenAI ChatGPT 的共同发明人 Diogo Almeida 创立）推出了 Jev。他们提出了完全相同的非自回归决策概念，仿佛这是一个全新的科学突破。但不同的是，他们发布时没有技术论文，没有开源权重，也没有任何开放的训练数据集。

My earlier model used PPO over sequence representations to output turn-by-turn conversion trajectories (probabilities from 0.0 to 1.0) in vertical sales conversations. Jev generalized parallel sampling using what they called RLCD (Reinforcement Learning for Calibrated Decisions) to output confidence distributions and schema choices horizontally, charging $0.042 per million input tokens with typical response times around 150 ms. 我早期的模型在垂直销售对话中使用基于序列表示的 PPO（近端策略优化）来输出逐轮转化轨迹（0.0 到 1.0 的概率）。Jev 则通过他们所谓的 RLCD（校准决策强化学习）将并行采样泛化，以水平方式输出置信度分布和模式选择，收费为每百万输入 token 0.042 美元，典型响应时间约为 150 毫秒。

It is incredibly frustrating when something you poured your heart into for months as an open-source researcher gets overlooked because it was built for a vertical use case, while a funded lab packages the same core idea horizontally and gets all the glory. But that is the open-source story in general 🙂. 当作为一名开源研究者，你倾注数月心血构建的东西因为是针对垂直用例而被忽视，而一家有资金的实验室将同样的核心理念进行水平封装并获得所有赞誉时，这令人极其沮丧。但这就是开源界的常态 🙂。

Instead of staying bitter, I decided to take everything I learned from my March 2025 and September 2025 papers, fix every architectural limitation of the old approach, and build a completely open, horizontal System 1 decision model: RL Agent. And because we built it properly on a bidirectional encoder, our model runs in 33 to 38 milliseconds on a GPU, making it roughly 4x faster than Jev’s published 150 ms latency, and it is 100% open-source. 与其耿耿于怀，我决定利用从 2025 年 3 月和 9 月论文中学到的一切，修复旧方法的所有架构局限，并构建一个完全开源、水平化的“系统 1”决策模型：RL Agent。由于我们基于双向编码器进行了正确构建，我们的模型在 GPU 上运行仅需 33 到 38 毫秒，比 Jev 公布的 150 毫秒延迟快了约 4 倍，而且它是 100% 开源的。

1. The Real Problem: Why LLMs are Terrible for Decisions

1. 真正的问题：为什么大语言模型不适合做决策

Every modern AI pipeline has a giant bottleneck: we use generative LLMs for simple reflex decisions. When a customer support ticket arrives, or an email hits your inbox, or a user submits a prompt to your API, you usually only need to answer a few simple questions: Which department should this go to? Is this email a phishing attack? Is this prompt trying to jailbreak the system? How urgent is this issue on a scale of 0 to 3? 每个现代 AI 流水线都有一个巨大的瓶颈：我们使用生成式大语言模型来处理简单的反射性决策。当客户支持工单到达、电子邮件进入收件箱，或用户向你的 API 提交提示词时，你通常只需要回答几个简单的问题：这应该分发到哪个部门？这封邮件是钓鱼攻击吗？这个提示词是否试图越狱系统？按 0 到 3 的等级划分，这个问题有多紧急？

Calling an 8B or 70B generative LLM for this is complete overkill. You wait 500ms to 2,000ms for tokens to stream out, spend real money on inference, and then have to write regex or JSON parsers to extract a clean label from free-form text. Worst of all, LLMs love to hallucinate and generate fake confidence. When an LLM outputs “confidence: 0.95”, it is just predicting tokens that sound confident. There is zero mathematical calibration behind it. 为此调用 8B 或 70B 的生成式大语言模型完全是杀鸡用牛刀。你需要等待 500 毫秒到 2000 毫秒来等待 token 流出，花费真金白银进行推理，然后还得编写正则表达式或 JSON 解析器从自由文本中提取干净的标签。最糟糕的是，大语言模型喜欢产生幻觉并生成虚假的置信度。当大语言模型输出“置信度：0.95”时，它只是在预测听起来自信的 token，背后没有任何数学上的校准。

We needed a model that works like the human brain’s System 1: instant, reflex decisions with honest, calibrated probabilities, taking only 30 to 40 milliseconds on standard hardware. 我们需要一个像人类大脑“系统 1”那样工作的模型：即时、反射性的决策，具备诚实且经过校准的概率，在标准硬件上仅需 30 到 40 毫秒。

2. The Three Decision Primitives

2. 三种决策原语

Following the System 1 philosophy, RL Agent accepts a state (raw text, email, ticket, or JSON document) along with one or more typed questions, and evaluates all of them in a single, parallel forward pass. It uses three primitives: 遵循“系统 1”哲学，RL Agent 接收一个状态（原始文本、电子邮件、工单或 JSON 文档）以及一个或多个类型化问题，并在单次并行前向传递中评估所有问题。它使用三种原语：

• choice: Pick one option from a dictionary of criteria. Returns the chosen label, probabilities for each candidate option, and a confidence score. (Great for department routing, intent detection, topic classification).
  • choice（选择）：从标准字典中选择一个选项。返回选定的标签、每个候选选项的概率以及置信度分数。（非常适合部门路由、意图检测、主题分类）。
• score: Place the state on an ordinal rubric like levels 0, 1, 2, 3. Returns an expected score value, probabilities across levels, and confidence. (Great for customer frustration, urgency, prompt harm severity).
  • score（评分）：将状态置于 0、1、2、3 等序数准则上。返回期望分值、各等级的概率以及置信度。（非常适合客户沮丧程度、紧急程度、提示词危害严重性）。
• noul: A direct boolean question returning calibrated probability P(true) from 0.0 to 1.0. (Great for detecting phishing, spam, jailbreaks, or churn risk).
  • noul（布尔判断）：一个直接的布尔问题，返回 0.0 到 1.0 之间经过校准的 P(true) 概率。（非常适合检测钓鱼、垃圾邮件、越狱或流失风险）。

Because the output space is strictly probabilities and numbers, the model never generates text, cannot hallucinate, and broken JSON is physically impossible. 由于输出空间严格限制为概率和数字，模型从不生成文本，不会产生幻觉，且不可能出现损坏的 JSON。

3. Model Architecture: 421M Parameters

3. 模型架构：4.21 亿参数

In my March 2025 work, I used frozen sequence embeddings combined with a separate PPO value network. It worked for turn-by-turn sales prediction, but it was not end-to-end and could not handle dynamic new questions at runtime. For RL Agent, we built a 421M-parameter end-to-end architecture with two tightly coupled components: 在我 2025 年 3 月的工作中，我使用了冻结的序列嵌入结合独立的 PPO 值网络。它适用于逐轮销售预测，但不是端到端的，且无法在运行时处理动态的新问题。对于 RL Agent，我们构建了一个 4.21 亿参数的端到端架构，包含两个紧密耦合的组件：

[Diagram/Structure omitted for brevity] [图表/结构略]