Cheaper LLM labelling / 更经济的 LLM 标注方案

I have a small project where I needed to label commits as either “maintenance” or “new development”. The obvious way to do it is with a cheap but relatively capable LLM, like GPT-5.6 Luna. I tested it on a small set of commits and manually verified its labelling, and it emitted the same label as I would have for the entire test set. That was good enough for me to roll out on a wider scale.

我有一个小项目，需要将代码提交（commits）标记为“维护（maintenance）”或“新开发（new development）”。最显而易见的方法是使用一个廉价但相对强大的大语言模型（LLM），比如 GPT-5.6 Luna。我在一小部分提交上进行了测试并手动验证了其标注结果，它给出的标签与我在整个测试集上的判断完全一致。这对我来说已经足够将其推广到更大规模的应用中了。

If we have Simon Willison’s llm CLI tool installed (and you should – it’s great!), we can call it in a pipe from Perl, and read its response. My script had a loop that retried the request a few times. I have experience of models sometimes failing to heed output format instructions, which is usually solved by retrying once or twice. My loop bailed at five attempts, and I’m not sure it was ever needed. Luna is a more capable model than those that have had trouble following output format instructions, but without that bookkeeping, the code for this is simple enough.

如果我们安装了 Simon Willison 的 llm 命令行工具（你应该安装它——它非常棒！），我们就可以在 Perl 中通过管道调用它并读取响应。我的脚本包含一个重试请求的循环。根据我的经验，模型有时会忽略输出格式指令，通常重试一两次就能解决。我的循环在尝试五次后会放弃，但我并不确定是否真的需要这么多次。Luna 比那些难以遵循输出格式指令的模型更强大，但即便没有这些防错机制，代码本身也足够简单。

In[1]:
sub classify {
    my ($msg) = @_;
    my $pid = open2( my $output, my $prompt, 'llm -m openrouter/openai/gpt-5.6-luna' );
    print $prompt prompt_template($msg);
    close $prompt;
    # Slurp the entire response and chomp off the trailing newline.
    chomp(my $result = do { local $/; <$output> });
    waitpid($pid, 0);
    return $result;
}

The drawback of this approach is that I wanted to label some 21,000 commits, and this shells out to the LLM for every single commit. Since the prompt is designed to contain mainly input tokens and not many output tokens, these calls are cheap in money, but the latency is no fun, at around 1.5 seconds per call. If I actually cared about latency, I would not use Perl to call a Python CLI that asks OpenRouter to send my request to OpenAI, but make the request directly to OpenAI. In this article I will pretend to care about latency, but it’s really more about sharing this cool technique.

这种方法的缺点在于，我需要标注大约 21,000 个提交，而这种方式对每一个提交都要调用一次外部 LLM。由于提示词设计为主要包含输入 Token 而输出 Token 很少，这些调用的金钱成本很低，但延迟令人难以忍受，每次调用大约需要 1.5 秒。如果我真的在意延迟，我就不会用 Perl 去调用一个 Python 命令行工具，再由它请求 OpenRouter 将我的需求转发给 OpenAI，而是会直接向 OpenAI 发起请求。在本文中，我假装很在意延迟，但其实更多是为了分享这个很酷的技巧。

Short-cutting with a faster classifier / 使用更快的分类器进行捷径处理

Some commits are obviously new development, or obviously maintenance. It would be nice if we could classify those through a more primitive – and faster – way, and only ask the LLM about the more difficult cases. Here’s the pseudo-code for such an algorithm:

有些提交显然属于新开发，或者显然属于维护。如果我们能通过一种更原始、更快速的方法来分类这些提交，而只在遇到困难情况时才询问 LLM，那就太好了。以下是该算法的伪代码：

In[2]:
sub labeling {
    my ($msg) = @_;
    # Take the word frequencies of the commit message as the features for the fast classifier.
    my $features = bag_of_words($msg);
    # Use the fast classifier when it has seen enough training samples, 
    # but skip it sometimes to prevent accidentally learning p=0 or p=1 
    # based on long runs of a single label.
    if ($fast_classifier->{samples} > 50 && rand() < 0.95) {
        my $p = $fast_classifier->predict($features);
        # Use the labels predicted by the fast classifier only if it is very confident.
        return 'maintenance' if $p < 0.08;
        return 'new development' if $p > 0.92;
    }
    # If we have insufficient samples or fall through, use the LLM classifier to get a label.
    my $label = classify($msg);
    # Since we got a real label, use it to train the fast classifier.
    $fast_classifier->refine($label, $features);
    return $label;
}

The question is what object implements the interface we have assumed for the fast classifier. We have called two non-trivial methods on it: predict($features) (return a probabilistic prediction) and refine($label, $features) (add the association to the training set). An obvious choice is naïve Bayes. The drawback is that it assumes independence between features, which generally won’t be the case for us. Another alternative is a logistic regression.

问题在于，哪个对象实现了我们为快速分类器假设的接口？我们调用了两个非平凡的方法：predict($features)（返回概率预测）和 refine($label, $features)（将关联关系添加到训练集）。一个显而易见的选择是朴素贝叶斯（Naïve Bayes）。其缺点是它假设特征之间相互独立，而这对我们来说通常是不成立的。另一个选择是逻辑回归（Logistic Regression）。

Streaming training for logistic regression / 逻辑回归的流式训练

I learned for this project that a logistic regression can be trained by streaming it one label at a time, and using gradient descent to nudge its weights toward a better fit. I haven’t fully worked through the derivation, but this is not that complicated as far as these things go. The final result is just 3–10 lines of code depending on how you count. I wish I knew this years ago! It’s so elegant.

在本项目中，我了解到逻辑回归可以通过一次处理一个标签进行流式训练，并利用梯度下降法不断调整权重以获得更好的拟合效果。我还没有完全推导过整个过程，但就这类问题而言，这并不复杂。最终结果只需 3 到 10 行代码，具体取决于你怎么计算。我真希望几年前就知道这一点！它太优雅了。

In[3]:
package LR {
    sub new {
        my ($class, %opt) = @_;
        my $self = bless {%opt}, $class;
        $self->{weights} //= {};
        $self->{learning} //= 0.1;
        $self->{regularisation} //= 0;
        $self->{samples} = 0;
        return $self;
    }

    sub score {
        my ($self, $features) = @_;
        my $z = 0;
        $z += ($self->{weights}{$_} // 0) * $features->{$_} for keys %$features;
        return $z;
    }

    sub predict {
        my ($self, $features) = @_;
        return 1 / (1 + exp(-$self->score($features)));
    }

    sub refine {
        my ($self, $label, $features) = @_;
        my $dlldw = ($label eq $self->{positive} ? 1 : 0) - $self->predict($features);
        $self->{weights}{$_} = ($self->{weights}{$_} // 0) - $self->{learning} * -$dlldw * $features->{$_} for keys %$features;
        $self->{weights}{$_} -= $self->{regularisation} * $self->{weights}{$_} for keys %{$self->{weights}};
        $self->{samples}++;
    }
};

And that’s it! The pseudo-code from before becomes actual, working code once we instantiate this class and store it in $fast_classifier. It all fits in 40 lines of plain Perl. Let’s step back and see what happened: We have an expensive…

就是这样！一旦我们实例化这个类并将其存储在 $fast_classifier 中，之前的伪代码就变成了实际可运行的代码。所有这些代码加起来不到 40 行纯 Perl。让我们回顾一下发生了什么：我们有一个昂贵的……