There’s no point at which turning your brain off will work

“关掉大脑”的工作方式永远行不通

In early 2025, I started seeing people turn off their brain as they use LLMs. They would have an LLM take an action (summarize text, write some code, etc.), and just assume that it worked. This generally didn’t work in early 2025 and the result was often quite silly. 2025年初，我开始观察到人们在使用大语言模型（LLM）时会选择“关掉大脑”。他们会让LLM执行某项操作（如总结文本、编写代码等），然后直接默认结果是正确的。在2025年初，这种做法通常行不通，结果往往显得十分荒谬。

As LLMs have gotten better, I’ve seen more of this. Sometimes, people will try to get the LLM to write some code for them and basically just assume that it works. Sometimes there’s a human in the loop and, if the thing doesn’t work, they’ll ask the LLM to figure out the problem and solve it. Niklas Gruhn calls some variants of doing this being a meat proxy. 随着LLM性能的提升，这种现象反而更多了。有时，人们会让LLM为他们写代码，并默认代码可以直接运行。有时会有“人在回路”的机制，如果代码报错，他们会要求LLM找出问题并修复。Niklas Gruhn将这种做法的某些变体称为“肉身代理”（meat proxy）。

Being a for loop meat proxy works better than it did in early 2025 and the software I’ve tried that’s developed like this sometimes actually sort of works. Not well enough that I’d want to use it or that it’s successful, but I’m impressed at how effective being a meat proxy is in September 2026. 作为“for循环肉身代理”的效果比2025年初要好，我尝试过用这种方式开发出的软件，有时确实能勉强运行。虽然还没达到我愿意使用或称其为成功的程度，但2026年9月的今天，这种“肉身代理”模式的有效性确实令我印象深刻。

You could imagine LLMs improving enough that brain-off meat-proxy development produces average quality software in the foreseeable future, or even that LLMs improve enough that they produce great software without a human in the loop. Let’s say that happens. What reason is there for the company to employ the meat proxy? The company can just run the LLM in a loop and lay off the employee. There’s no point at which this methodology will work for the employee. 你可以想象，在可预见的未来，LLM的进步足以让“关掉大脑的肉身代理开发”产出平均水平的软件，甚至LLM可能进化到无需人类参与就能产出优秀的软件。假设这种情况真的发生了，公司还有什么理由雇佣“肉身代理”呢？公司完全可以直接让LLM循环运行，然后裁掉员工。对于员工来说，这种工作方法在任何阶段都行不通。

Thanks to Max Bittker, Yossi Kreinin, Luke Burton, Thomas Dullien, Dennis Snell, Peter Geoghegan, and Jamie Brandon for comments/corrections/discussion. I’ve been having this thought for about a year and a half now. I have it more frequently now as LLMs get better and I see people spend more time turning their brain off when interacting with LLMs. 感谢Max Bittker、Yossi Kreinin、Luke Burton、Thomas Dullien、Dennis Snell、Peter Geoghegan和Jamie Brandon提供的评论、更正和讨论。这个想法在我脑海中已经盘旋了一年半了。随着LLM的进步，以及我看到越来越多的人在与LLM交互时选择“关掉大脑”，这种想法出现的频率也越来越高。

---

Luke Burton had this comment: I think being able to do this says more about the type of work being done than people think. I will only walk away from work like this if the task is quite low value, if it can afford to fail. For high value tasks, the probability of an LLM one-shotting them is much lower. I have to assume the role of QA, engineering manager, and architect. The while loop often feels like a crunch time. I feel the nagging suspicion I’ve missed something and that a badly specified prompt could result in an architectural choice that needs to be undone. Luke Burton对此评论道：我认为能够做到这一点，反映出的工作性质比人们想象的要多。只有在任务价值很低、且允许失败的情况下，我才会对工作采取“甩手不管”的态度。对于高价值任务，LLM一次性完成的概率要低得多。我必须同时扮演质量保证（QA）、工程经理和架构师的角色。这种“while循环”式的过程往往让人感觉像是在赶工期。我总有一种挥之不去的疑虑，担心自己遗漏了什么，或者一个定义糟糕的提示词（prompt）可能导致需要推倒重来的架构决策。

Another observation is that the high throughput causes me to raise my own bar for what I ship. Whereas before I might have shipped an MVP and iterated, now I have agents polish and explore edge cases well beyond my norm, which they invariably fail to do unless prompted. Maybe it raises some uncomfortable thoughts for people, but my question for the meat proxies out there if the agents are nailing it so easily: 1) is it possible you’ve been coasting a bit already? 2) why aren’t you pushing agents well beyond tasks they can tackle so easily? 另一个观察是，高吞吐量促使我提高了交付标准。以前我可能会先发布一个最小可行性产品（MVP）再进行迭代，而现在我会让智能体（agents）去打磨并探索远超我常规预期的边缘情况——当然，除非我明确提示，否则它们总是做不到这一点。这可能会引发一些令人不安的思考，但对于那些“肉身代理”们，如果智能体真的能如此轻松地搞定任务，我的问题是：1）你是否已经在“摸鱼”了？2）为什么不尝试让智能体去挑战那些它们无法轻易解决的任务呢？

We’ve been doing something you’d think is extremely amenable to “hands off” automation, which is converting [redacted] to build with Bazel. It has taken us months even with agents. There’s a lot of intangible, hard-to-specify requirements buried inside this task and having agents walk that line means constant supervision. Giving them a prompt like “convert this to Bazel” and walking away is at minimum many months in the future, maybe years, and maybe not ever? There are too many decision points, and too many unknown unknowns involved. 我们一直在做一件你认为非常适合“全自动”的事情，那就是将[已脱敏]项目迁移到Bazel构建系统。即便有智能体辅助，我们也花了数月时间。这项任务中隐藏着许多难以言说、难以定义的具体需求，让智能体去处理这些细节意味着必须进行持续的监督。给它们一个“将其转换为Bazel”的提示词然后甩手不管，至少在未来几个月甚至几年内是不可能的，甚至永远都不可能实现。其中涉及太多的决策点和太多的“未知未知”（unknown unknowns）。

Like how often does this scenario come up: you encounter some code and it’s not clear why it functions this way, but knowing that materially changes what course of action you should take. Maybe it changes the dev experience, maybe you don’t know if some customer has started using it, so on and so forth. How exactly do you meat proxy your way through that? Conversely, you review what you’ve done with some stakeholder and they say “oh that? that part of it wasn’t needed, we aren’t even using that any more”. What kind of decisions got made around the false assumption that a certain element needed to be preserved? 比如这种情况经常发生：你遇到一段代码，不清楚它为什么这样运行，但了解这一点会实质性地改变你应采取的行动方案。也许这会改变开发体验，也许你不知道是否有客户已经开始使用它，等等。你究竟该如何通过“肉身代理”来解决这些问题？反过来，当你与利益相关者审查工作成果时，他们说：“哦，那个部分不需要了，我们已经不用了。”那么，当初基于“某个元素必须保留”这一错误假设所做出的决策，又算什么呢？

---

[End of Luke’s comment, comment from me]. A place where it’s more obvious you need to make decisions is when the agent runs into something that’s out of distribution. A minor version of this was when we compared how well agents use different programming languages and agents were much worse at obscure languages, which they’re trained on, just not as much as with mainstream languages. [Luke的评论结束，以下是我的评论]。当智能体遇到“分布外”（out of distribution）的情况时，你需要做出决策的需求就变得更加明显。一个轻微的例子是，当我们比较智能体使用不同编程语言的能力时，发现它们在冷门语言上的表现要差得多——虽然它们接受过这些语言的训练，但训练量远不及主流语言。

A more out of distribution example is if you try to play a board game (especially a modern game and not one of the classical games like chess or go). In general, for a game like Lost Cities or Dominion, a SOTA model and harness is worse than a human who’s reasonable at board games but has never played the game before. If you ask the agent about the game, it knows a lot about the game and can say things that sound like they make sense to someone who doesn’t understand the game, but are obviously wrong to anyone who does understand the game. 一个更典型的“分布外”例子是尝试玩桌游（尤其是现代桌游，而非国际象棋或围棋等经典游戏）。通常情况下，对于《失落的城市》（Lost Cities）或《领土》（Dominion）这类游戏，最先进（SOTA）的模型及其框架的表现，甚至不如一个虽然没玩过该游戏、但具备良好桌游基础的人类。如果你询问智能体关于游戏的问题，它能说出很多听起来头头是道、能糊弄外行的话，但在真正懂行的人看来，这些话显然是错误的。

I recently played some Dominion with a new player who thought that using ChatGPT to help them understand the game would help them learn and play the game. I was quite skeptical of this and suggested that it will probably make them worse (which, AFAICT, it did). After playing a few games, I looked at what ChatGPT was telling them, and it was maybe half right and half wrong, but the half wrong parts were steering them to a worse place than someone who generally plays games well and uses general game playing heuristics would do. 我最近和一位新玩家玩《领土》，他认为使用ChatGPT能帮助他理解并玩好游戏。我对此非常怀疑，并建议这可能会适得其反（据我观察，确实如此）。玩了几局后，我查看了ChatGPT给他的建议，发现大概是一半对一半错，但那些错误的部分将他引向了比“依靠通用游戏策略的玩家”更糟糕的境地。

BTW, there’s enough public information out there that I think that someone who’d never played before, but decided to spend, say, five hours reading about the game and seeing what information is out there, could easily be 99%-ile or above at the game if they did some pre-reading (maybe 30 minutes if using references while playing is allowed). I think that would be un-fun and I wouldn’t recommend that anyone do it, but given that agents can do searches, query APIs, etc., it shows you the gap between a human. 顺便说一句，公开信息已经足够丰富，我认为一个从未玩过该游戏的人，如果愿意花大约五个小时阅读相关资料，完全可以轻松达到该游戏玩家群体的前1%甚至更高水平（如果允许在游戏过程中查阅参考资料，可能只需30分钟）。我认为这样做会失去乐趣，我不建议任何人这样做，但考虑到智能体可以进行搜索、查询API等操作，这恰恰展示了人类与智能体之间的差距。