I built non-autoregressive decision models with RL a year ago
I built non-autoregressive decision models with RL a year ago
我在一年前就用强化学习构建了非自回归决策模型
Everyone in AI right now is talking about a new kind of model: an architecture that is not autoregressive, does not generate text, and gives lightning-fast probability predictions over structured schemas. 目前人工智能领域的所有人都在谈论一种新型模型：一种非自回归、不生成文本，且能针对结构化模式提供闪电般快速概率预测的架构。
Seeing the hype online feels both validating and deeply frustrating. I worked on this literally one year back in March 2025. I spent months of hard work, sweat, and sleepless nights building it, published an arXiv paper (arXiv:2503.23303), released the model weights on Hugging Face (sales-conversion-model-reinf-learning), published the open dataset (saas-sales-conversations), built a PyPI package, and posted the whole approach on Reddit (r/LocalLLaMA discussion). 看到网上的这些炒作，我既感到被认可，又深感沮丧。我早在 2025 年 3 月就着手研究这个课题了。我花费了数月的心血、汗水和无数个不眠之夜来构建它，发表了 arXiv 论文 (arXiv:2503.23303)，在 Hugging Face 上发布了模型权重 (sales-conversion-model-reinf-learning)，发布了开放数据集 (saas-sales-conversations)，构建了 PyPI 包，并将整个方法发布在了 Reddit 上 (r/LocalLLaMA 讨论区)。
Then in September 2025, I published a second paper (arXiv:2510.01237), formalizing the framework for schema-based decisions guided by reinforcement learning. The guiding brain in my system was always reinforcement learning, not just an embedding model or an autoregressive LLM. 随后在 2025 年 9 月，我发表了第二篇论文 (arXiv:2510.01237)，将由强化学习引导的基于模式的决策框架进行了形式化。我系统中的核心大脑始终是强化学习，而不仅仅是一个嵌入模型或自回归大语言模型 (LLM)。
And then in September 2026, a well-funded frontier lab called TypeSafe AI (founded by Diogo Almeida, a co-inventor of ChatGPT at OpenAI) launched Jev. They proposed the exact same non-autoregressive decision concept as if it was a brand-new scientific breakthrough. Except they launched without technical papers, without open weights, and with zero open training datasets. 接着在 2026 年 9 月，一家资金雄厚的前沿实验室 TypeSafe AI（由 OpenAI ChatGPT 的共同发明人 Diogo Almeida 创立）推出了 Jev。他们提出了完全相同的非自回归决策概念，仿佛这是一个全新的科学突破。但不同的是，他们发布时没有技术论文，没有开放权重，也没有任何开放的训练数据集。
My earlier model used PPO over sequence representations to output turn-by-turn conversion trajectories (probabilities from 0.0 to 1.0) in vertical sales conversations. Jev generalized parallel sampling using what they called RLCD (Reinforcement Learning for Calibrated Decisions) to output confidence distributions and schema choices horizontally, charging $0.042 per million input tokens with typical response times around 150 ms. 我早期的模型在垂直销售对话中使用 PPO（近端策略优化）处理序列表示，以输出逐轮的转化轨迹（概率从 0.0 到 1.0）。Jev 则通过他们所谓的 RLCD（用于校准决策的强化学习）将并行采样泛化，以水平方式输出置信度分布和模式选择，收费标准为每百万输入 token 0.042 美元，典型响应时间约为 150 毫秒。
Instead of staying bitter, I decided to take everything I learned, fix every architectural limitation of the old approach, and build a completely open, horizontal System 1 decision model family: Laya. 与其耿耿于怀，我决定利用我所学到的一切，修复旧方法的所有架构局限性，并构建一个完全开放、水平化的“系统 1”决策模型家族：Laya。
And because we built it properly on bidirectional encoders, our models run in 32.8 milliseconds on a single GPU (7.2 ms/question batched), making it 6 to 8 times faster than Jev, with full support for over 100 languages, zero API subscription costs, and 100% open-source Apache 2.0 weights. 由于我们基于双向编码器正确地构建了它，我们的模型在单块 GPU 上运行仅需 32.8 毫秒（批处理时为 7.2 毫秒/问题），这使其比 Jev 快 6 到 8 倍，并全面支持 100 多种语言，零 API 订阅成本，且拥有 100% 开源的 Apache 2.0 权重。
1. The Core Realization: System 1 vs System 2
1. 核心认知：系统 1 与系统 2
Every modern AI pipeline has a giant bottleneck: we use generative LLMs for simple reflex decisions. 每个现代 AI 流水线都有一个巨大的瓶颈：我们使用生成式 LLM 来处理简单的反射性决策。
When a customer support ticket arrives, or an email hits your inbox, or a user submits a prompt to your API, you usually only need to answer simple, structured questions: 当客户支持工单到达、电子邮件进入收件箱，或用户向你的 API 提交提示词时，你通常只需要回答简单、结构化的问题：
- Which department should this ticket route to? (该工单应路由到哪个部门？)
- Is this incoming email a phishing attack or spam? (这封邮件是钓鱼攻击还是垃圾邮件？)
- Is this prompt trying to jailbreak or inject instructions? (该提示词是否试图越狱或注入指令？)
- How urgent is this issue on an ordinal rubric (0 to 3)? (按序数标准（0 到 3）衡量，此问题有多紧急？)
- Does this query require code execution or a simple factual reply? (此查询需要代码执行还是简单的事实回复？)
Calling an 8B, 70B, or frontier generative LLM for this is complete overkill. You wait 500 ms to 2,000 ms for tokens to stream out, spend real money on inference, and then have to write regex or JSON parsers to extract a clean label from free-form text. Worst of all, LLMs love to hallucinate and generate fake confidence. When an LLM outputs “confidence: 0.95”, it is just predicting tokens that sound confident. There is zero mathematical calibration behind it. 为此调用 8B、70B 或前沿生成式 LLM 完全是杀鸡用牛刀。你需要等待 500 到 2000 毫秒让 token 流出，花费真金白银进行推理，然后还得编写正则表达式或 JSON 解析器从自由文本中提取干净的标签。最糟糕的是，LLM 喜欢产生幻觉并生成虚假的置信度。当 LLM 输出“置信度：0.95”时，它只是在预测听起来自信的 token，其背后没有任何数学校准。
We needed a model that works like the human brain’s System 1: instant reflex decisions with honest, calibrated probabilities, taking only 30 to 35 milliseconds on standard commodity hardware. 我们需要一个像人类大脑“系统 1”那样工作的模型：即时反射性决策，具备诚实、经过校准的概率，且在标准商用硬件上仅需 30 到 35 毫秒。
2. The Three Decision Primitives
2. 三个决策原语
Laya evaluates typed questions over any state (raw text, email, ticket, or JSON document) in a single forward pass. It relies on three primitives: Laya 在单次前向传递中评估针对任何状态（原始文本、电子邮件、工单或 JSON 文档）的类型化问题。它依赖于三个原语：
- choice: Pick one option from a dictionary of criteria. Returns the selected key, probability distribution across all options, and a calibrated confidence score. (从标准字典中选择一个选项。返回选定的键、所有选项的概率分布以及校准后的置信度分数。)
- score: Place the state on an ordinal rubric (levels 0, 1, 2, …). Returns the expected level, the distribution over rubric ranks, and confidence. (将状态置于序数标准（0, 1, 2… 级）上。返回预期的级别、等级分布和置信度。)
- noul: A direct boolean question returning calibrated probability P(true) from 0.0 to 1.0 (with P(false) = 1 - P(true) by construction). (一个直接的布尔问题，返回从 0.0 到 1.0 的校准概率 P(true)（根据构造，P(false) = 1 - P(true)）。)
Because the output space consists purely of probabilities and numbers, the model never generates text, cannot hallucinate, and schema violations or malformed JSON are physically impossible. 由于输出空间完全由概率和数字组成，模型从不生成文本，不会产生幻觉，且模式违规或格式错误的 JSON 在物理上是不可能的。
3. The Three Checkpoints & Bundled Hub Architecture
3. 三个检查点与捆绑式 Hub 架构
One model cannot be optimal for every task and language. We released three specialized checkpoints, now consolidated under a single repository hub on Hugging Face: 没有一个模型能对所有任务和语言都达到最优。我们发布了三个专门的检查点，现在统一整合在 Hugging Face 的一个存储库中心下：
| Checkpoint | Backbone Encoder | Params | Context | Primary Strength |
|---|---|---|---|---|
| convaiinnovations/laya | ModernBERT-large | 421M | 512 | English text classification, guardrails, email triage |
| convaiinnovations/laya-multilingual | mmBERT-base (256k vocab) | 322M | 1024 (up to 8k) | 100+ languages, 2.2x faster, cross-lingual NLI |
| convaiinnovations/laya-typed-decisions | ModernBERT-large | 421M | 1024 | Agent observability, customer service, invoice processing, security alerts (0.766 acc) |
Selective Subfolder Downloads 选择性子文件夹下载
Rather than forcing users to manage three separate repositories or download 2.5 GB of combined weights, the main repository convaiinnovations/laya bundles all three. Using Hugging Face’s allow_patterns, Laya’s SDK downloads only the specific subfolder requested: 为了不强迫用户管理三个独立的存储库或下载 2.5 GB 的组合权重，主存储库 convaiinnovations/laya 将三者捆绑在一起。利用 Hugging Face 的 allow_patterns，Laya 的 SDK 仅下载请求的特定子文件夹：
# Downloads English model (~808 MB)
agent_en = laya.load("convaiinnovations/laya")
# Downloads ONLY the multilingual subfolder (~647 MB), not the entire 2.5 GB bundle
agent_ml = laya.load("convaiinnovations/laya", subfolder="multilingual")4. Why Routing Is Essential: The Multi-Script Reality
4. 为什么路由至关重要：多脚本现实
One of the most eye-opening findings from our 51-language sweep on the MASSIVE benchmark (20 options, random baseline = 0.050) was how English models fail outside Latin script. 我们在 MASSIVE 基准测试（20 个选项，随机基准 = 0.050）上进行的 51 种语言扫描中，最令人大开眼界的发现之一是英语模型在拉丁字母之外的表现是多么糟糕。
ModernBERT-large’s 50,000-token English BPE vocabulary simply shreds non-Latin alphabets: ModernBERT-large 的 50,000 个 token 的英语 BPE 词汇表完全无法处理非拉丁字母：
- Khmer: 0.000 accuracy at 0.952 mean confidence. Not one correct decision in 100 questions, while reporting ~95% confidence. (高棉语：准确率为 0.000，平均置信度为 0.952。100 个问题中没有一个正确决策，却报告了约 95% 的置信度。)
- Armenian: 0.050 accuracy (exact coin-flip random) at 0.885 confidence. (亚美尼亚语：准确率为 0.050（完全是抛硬币的随机水平），置信度为 0.885。)
- Hebrew: 0.060 accuracy at 0.964 (希伯来语：准确率为 0.060，置信度为 0.964。)