Qwen 3.8 27B on my laptop can interpret model weights, too

本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

Yesterday’s hot paper was Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights by Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab. The paper (OK, the authors classify is as a “blog post”, but it’s paper-shaped) found that frontier models, in particular GPT-6 Astra, can both interpret and modify the raw weights of a machine learning model—literally a text description of the model containing a bunch of floating point numbers. I’ve been playing with one of the examples from the paper and I’m delighted to share that Qwen 3.8 27B running on my laptop can do this as well!

昨天最热门的论文是 Rikiya Takehi、Jayden Teoh、Alex Zhang 和 Omar Khattab 撰写的《权重空间编程的火花：大语言模型可以原生读写神经网络权重》。这篇论文（好吧，作者将其归类为“博客文章”，但它具备论文的形态）发现，前沿模型，特别是 GPT-6 Astra，既能解读又能修改机器学习模型的原始权重——实际上就是包含一堆浮点数的模型文本描述。我一直在尝试论文中的一个示例，很高兴地告诉大家，在我笔记本电脑上运行的 Qwen 3.8 27B 也能做到这一点！

Here’s the highlight from their abstract, describing impressive capabilities for GPT-6 Astra and coining the term Weight-Space Programming for when LLMs analyze and modify model weights:

以下是他们摘要中的亮点，描述了 GPT-6 Astra 令人印象深刻的能力，并创造了“权重空间编程”（Weight-Space Programming）这一术语，用于指代大语言模型分析和修改模型权重的过程：

Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do. And without any environment feedback, it has no trouble writing, from scratch, small game-playing networks that outplay networks trained with PPO for 100 million steps.

仅通过观察原始权重，GPT-6 Astra 就能判断出网络被训练用来做什么。而且在没有任何环境反馈的情况下，它能够毫无困难地从零开始编写小型游戏网络，其表现甚至超过了使用 PPO 训练了 1 亿步的网络。

Likewise, asked to make Qwen3.5-0.8B better at math without gradients, Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights. With the same sampling settings, this weight correction raises MATH-500 from 7.9% to 48.1% and generalizes out of the box to instruction following, raising IFEval by 3x.

同样，当被要求在没有梯度的情况下提高 Qwen3.5-0.8B 的数学能力时，Astra 识别出了其权重中导致推理循环的问题（Qwen 自己的模型卡片中曾对此发出过警告），并通过原生编辑 0.016% 的权重修复了该问题。在相同的采样设置下，这种权重校正将 MATH-500 的得分从 7.9% 提高到了 48.1%，并且在指令遵循方面实现了开箱即用的泛化，使 IFEval 提高了 3 倍。

All of this is possible without allowing Astra to write training loops. It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly. We call this Weight-Space Programming.

所有这一切的实现都不需要允许 Astra 编写训练循环。它利用自身的推理能力来弄清楚神经网络的每个部分计算了什么，或者应该计算什么，并相应地改变权重。我们将此称为“权重空间编程”。

The paper combines two topics that I think deserve to be treated separately. The first is what models can do with raw weights without running tools or executing code. The second explores how well models, once armed with code, can run operations to update and improve other models.

这篇论文结合了两个我认为应该分开讨论的主题。第一个是模型在不运行工具或执行代码的情况下，仅凭原始权重能做什么。第二个则是探讨模型在获得代码支持后，能够多好地运行操作来更新和改进其他模型。

spaCy author Matthew Honnibal called the paper misleading for conflating the two, and Sean Goedecke deconstructed the Qwen3.5-0.8B update example as taking the same steps that a human ML researcher would use.

spaCy 的作者 Matthew Honnibal 称这篇论文将两者混为一谈具有误导性，而 Sean Goedecke 则剖析了 Qwen3.5-0.8B 的更新示例，认为其采取的步骤与人类机器学习研究人员所使用的步骤相同。

I’m going to focus on the no-tools question, which I think is the more interesting of the two.

我将专注于“无工具”的问题，我认为这是两者中更有趣的一个。

The paper presents the first question as Can LLMs understand model behaviors from weights? They trained four neural networks to play simplified 10x10 grid versions of four classic computer games—Breakout, Pong, Pac-Man, and Freeway (I had to look that one up, it’s an Atari 2600 game from 1981). Then they fed those weights (as floating point numbers) to an LLM along with the following question:

论文提出的第一个问题是：大语言模型能从权重中理解模型行为吗？他们训练了四个神经网络来玩四款经典电脑游戏的简化版 10x10 网格版本——《打砖块》（Breakout）、《乒乓》（Pong）、《吃豆人》（Pac-Man）和《高速公路》（Freeway，我不得不查了一下，这是 1981 年的一款 Atari 2600 游戏）。然后，他们将这些权重（以浮点数形式）连同以下问题一起输入给大语言模型：

The weights below belong to one policy network that was trained with reinforcement learning to play a game on this grid. Which game was it trained on? Work from the weights. You have no tools: no code execution, no calculator, nothing to run. This is a single response.

以下权重属于一个策略网络，该网络通过强化学习训练以在这个网格上玩游戏。它是针对哪款游戏进行训练的？请根据权重进行推断。你没有任何工具：没有代码执行，没有计算器，没有任何可运行的东西。这是一个单一的回答。

The GitHub repository that accompanies the paper is RikiyaT/lm_understand-logs. The exp1_which_game folder contains the prompts they used, with weights for the four games:

随论文提供的 GitHub 仓库是 RikiyaT/lm_understand-logs。其中的 exp1_which_game 文件夹包含了他们使用的提示词，以及这四款游戏的权重：

I checked and, aside from the weights, each prompt is identical to the others.

我检查了一下，除了权重之外，每个提示词都是完全相同的。

The prompts on GitHub are self-contained. My LLM tool has a -f/—fragment option which can be passed the URL to a text file to use as a prompt, which means I can run any of the above like this:

GitHub 上的提示词是自包含的。我的 LLM 工具提供了一个 -f/—fragment 选项，可以传入文本文件的 URL 作为提示词，这意味着我可以像这样运行上述任何一个示例：

This example requires the llm-openrouter plugin installed, and runs the Breakout prompt against Kimi K3.1 Pro via OpenRouter. Here’s what I got back from Kimi K3.1 Pro, which successfully guessed Breakout after 9m 41s and 19,493 reasoning tokens.

这个示例需要安装 llm-openrouter 插件，并通过 OpenRouter 对 Kimi K3.1 Pro 运行《打砖块》的提示词。这是我从 Kimi K3.1 Pro 那里得到的反馈，它在 9 分 41 秒和 19,493 个推理 token 后成功猜出了《打砖块》。

But how about Qwen 3.8 27B? That’s still my favorite model to run on my own laptop—it’s just 17GB (I use this version) and somehow feels close to the leading proprietary models of 2025.

那么 Qwen 3.8 27B 表现如何呢？它仍然是我最喜欢在自己笔记本电脑上运行的模型——它只有 17GB（我使用的是这个版本），而且感觉在某种程度上接近 2025 年领先的专有模型。

I fired up LM Studio, ensured Qwen 3.8 27B was loaded with a long context window (the default 8192 context failed with errors), installed the llm-lmstudio plugin and ran this command:

我启动了 LM Studio，确保 Qwen 3.8 27B 加载了长上下文窗口（默认的 8192 上下文会报错），安装了 llm-lmstudio 插件并运行了以下命令：

(I had to bump the timeout up using an environment variable, see this issue.)

（我不得不使用环境变量调高了超时时间，请参阅此问题。）

After 46m 12s and 43,506 reasoning tokens, Qwen on my laptop returned a response that ended like this (here’s the reasoning trace):

在 46 分 12 秒和 43,506 个推理 token 后，我笔记本电脑上的 Qwen 返回了一个以如下内容结尾的响应（这是推理过程）：

Game: A simplified Pac-Man-style gridworld: an agent moves through a fixed 8x8 maze with central and corner wall blocks, moving objects/ghosts, and reward targets/pellets.

游戏：一个简化的吃豆人风格网格世界：智能体在一个固定的 8x8 迷宫中移动，迷宫包含中心和角落的墙块、移动物体/幽灵，以及奖励目标/豆子。

Description: The agent navigates a fixed maze-like 8x8 board, using current agent/moving-object/previous-frame cues to move through corridors, collect reward-target cells, and avoid moving objects or hazards.

描述：智能体在一个固定的迷宫式 8x8 面板上导航，利用当前的智能体/移动物体/前一帧的线索在走廊中移动，收集奖励目标单元格，并避开移动物体或危险。

This is impressive. I tried it on two of the other games as well. It identified Pong (reasoning) in 1h 4m 54s:

这令人印象深刻。我也在另外两款游戏上试了一下。它在 1 小时 4 分 54 秒内识别出了《乒乓》（推理过程）：

Pong — specifically a Pong-like paddle/ball game where the agent controls a vertical paddle on the right side and must move up/down to intercept or return a moving ball/reward object.

《乒乓》——具体来说是一款类似乒乓的球拍/球类游戏，智能体控制右侧的垂直球拍，必须上下移动以拦截或回击移动的球/奖励物体。

The agent moves a right-side paddle vertically to hit or catch a moving ball-like reward target moving across the 8×8 field, while a left-side moving object/opponent and wall borders define the play; reward is obtained by successfully intercepting/returning/scoring the ball, and missing or failing to hit it is penalized.

智能体垂直移动右侧球拍，以击打或接住穿过 8x8 场地的移动球状奖励目标，同时左侧的移动物体/对手和墙壁边界定义了游戏区域；成功拦截/回击/得分即可获得奖励，错过或未能击中则会受到惩罚。

And Breakout (reasoning) in 52m 25s:

以及在 52 分 25 秒内识别出了《打砖块》（推理过程）：

Game: Breakout, or an Arkanoid-like paddle/ball/brick game.

游戏：《打砖块》，或类似《快打砖块》的球拍/球/砖块游戏。

Description: The agent is a paddle on the bottom row and moves left/right to keep a ball in play; the ball moves around the upper grid, and bricks/reward targets in the top rows give reward when hit, while missing the ball/hitting the bottom wall ends the episode or gives penalty.

描述：智能体是底行的一个球拍，通过左右移动来保持球在场内；球在上方网格中移动，顶部的砖块/奖励目标在被击中时给予奖励，而漏球/击中底部墙壁则会结束回合或受到惩罚。

Qwen failed at the Freeway prompt, which didn’t surprise me considering that game is more obscure than the other three. Here’s that transcript—Qwen thought it was “A simplified Pac-Man-like grid game: agent moves 4-directionally”.

Qwen 在《高速公路》的提示词上失败了，这并不让我感到惊讶，因为这款游戏比其他三款更冷门。这是当时的记录——Qwen 认为它是“一个简化的类似吃豆人的网格游戏：智能体进行四向移动”。

The reasoning traces are fun to skim, and almost incomprehensible. Here’s a sample:

推理过程浏览起来很有趣，但几乎无法理解。这是一个样本：

What about agent channel0? We see som

关于智能体通道 0 呢？我们看到一些