Build your own decision model

“System one” decision models are models that infer and respond with calibrated probabilities or every allowed answer.

“系统一”决策模型是指那些能够推断并以校准概率或针对每个允许的答案进行响应的模型。

Consider your everyday language model, to get typed output from it (JSON), you may use Structured Output to constrain the output to guaranteed valid JSON. While model prefills the input in one pass, it still has to go perform a pass for every token in order to generate a valid response.

以你日常使用的语言模型为例，为了从中获得类型化的输出（JSON），你可以使用结构化输出（Structured Output）来将输出限制为保证有效的 JSON。虽然模型会在一次传递中预填充输入，但它仍然必须为每个标记执行一次传递，以便生成有效的响应。

In this example, 11 passes are required to generate the final output. (We’re not accounting for speculative decoding and other inference optimization techniques.)

在此示例中，生成最终输出需要 11 次传递。（我们未考虑推测性解码和其他推理优化技术。）

Decision models such as Jev, make the assumption that there are fixed options we can select from and we can do so quickly by making a single pass. In this example we constrain the set of possible outputs to the options A, B, C, D, E. By masking other items in the vocabulary, the model can only emit those tokens. By selecting the highest probability output, we get our answer.

诸如 Jev 之类的决策模型假设存在我们可以选择的固定选项，并且我们可以通过单次传递快速完成选择。在此示例中，我们将可能的输出集限制为选项 A、B、C、D、E。通过屏蔽词汇表中的其他项目，模型只能发出这些标记。通过选择概率最高的输出，我们就能得到答案。

Since the outputs are constrained to only a fixed set of options, the model can’t select anything outside of those. This however does not guarantee that the output will be correct. It’s also common to treat the output token probabilities a confidence scores in this context, but without additional training, those scores likely reflect its confidence in what the next token will be rather than the true probability of the response being the correct answer.

由于输出仅限于一组固定的选项，模型无法选择这些选项之外的任何内容。然而，这并不能保证输出是正确的。在这种情况下，将输出标记概率视为置信度分数也很常见，但如果没有额外的训练，这些分数很可能反映的是模型对下一个标记是什么的置信度，而不是响应作为正确答案的真实概率。

We can emulate this behavior by constraining output tokens using an LLM. Here I’m using Qwen/Qwen3-1.7B

我们可以通过使用大语言模型（LLM）限制输出标记来模拟这种行为。这里我使用的是 Qwen/Qwen3-1.7B。

Running it with an simple question to test it yeilds the following output

用一个简单的问题来测试它，得出了以下输出。

The model was able to make sense of our input and make a prediction that reasonably corresponds to the correct answer.

该模型能够理解我们的输入，并做出与正确答案合理对应的预测。

We can test the accuracy of the model by running it against public datasets. I ran this against a random sample holdout of CommonsenseQA

我们可以通过在公共数据集上运行模型来测试其准确性。我针对 CommonsenseQA 的随机保留样本运行了此测试。

Not bad for a 1.7B model, Running a quick finetune on the dataset gives us slightly better performance

对于一个 1.7B 的模型来说还不错，在数据集上进行快速微调可以获得稍好的性能。

Testing the model against a very ambiguous problem demonstrates an interesting problem.

针对一个非常模棱两可的问题测试该模型，揭示了一个有趣的问题。

There should be no clear answer here, but treating the output probabilities as a pseudo “confidence” score, shows that the model is extremely overconfident in this answer.

这里本不应该有明确的答案，但将输出概率视为伪“置信度”分数，表明模型对这个答案极其过度自信。

If we bin the confidence score ranges in the eval I ran earlier, we can see that the model’s confidence does not match its accuracy. This means that the model is not calibrated.

如果我们对我之前运行的评估中的置信度分数范围进行分箱，我们可以看到模型的置信度与其准确性不匹配。这意味着模型没有经过校准。

We can notice that the model tends to be extremely overconfident in the 0.9 - 1.0 bin but it’s only correct 70% of the time. When it makes a prediction with 0.8 - 0.9 confidence it’s only accurate ~40% of the time. This means that the model is generally overconfident in its predictions.

我们可以注意到，模型在 0.9 - 1.0 的区间内往往极其过度自信，但其正确率仅为 70%。当它以 0.8 - 0.9 的置信度进行预测时，准确率仅为约 40%。这意味着模型在预测时通常过于自信。

Since our goal is to have the model output scores that is reflective of its accuracy, one method we can use to callibrate it is through temperature scaling. By modifying the temperature value, we can flatten its output probability distribution curve and scale it to approximate its accuracy.

由于我们的目标是让模型输出反映其准确性的分数，我们可以使用的一种校准方法是温度缩放（temperature scaling）。通过修改温度值，我们可以平滑其输出概率分布曲线，并对其进行缩放以逼近其准确性。

Curve fitting fit the temperature parameter to the model’s accuracy, I found 3.797280788421631 as a temp value.

通过曲线拟合将温度参数与模型的准确性相匹配，我发现 3.797280788421631 是一个合适的温度值。

This gets us a much better calibration. If you want to play around with this, I made a GitHub repo with scripts that walk you through building a dataset, evaluating, finetuning and calibrating your own model. I encourage pulling it and trying it on other bigger models.

这使我们获得了更好的校准效果。如果你想尝试一下，我创建了一个 GitHub 仓库，其中包含引导你构建数据集、评估、微调和校准你自己的模型的脚本。我鼓励你拉取代码并在其他更大的模型上进行尝试。