Yap: a particular kind of slop
Yap: a particular kind of slop
Yap:一种特殊的“废话”
yap (verb) to talk in a shrill insistent way yap(动词):以尖锐且喋喋不休的方式说话。
Comments
关于注释
Writing code comments has always been hard, because communicating is hard. It’s hard to not assume context. It’s hard to use the right terminology consistently. It’s hard to even put comments in the right spots to begin with. It generally takes me several iterations of rewording a comment to get something I’m happy with. I’ll write something lousey that takes 5 times as long to read as it should, and then read it. Then I’ll remove a couple sentences, tweak what remains and do it again. 编写代码注释一直以来都很困难,因为沟通本身就很难。很难做到不预设背景,很难始终如一地使用正确的术语,甚至连把注释放在合适的位置都很困难。通常我需要反复修改好几次才能写出让自己满意的注释。我会先写出一段糟糕的文字,读起来比它应有的长度长上五倍,然后再读一遍。接着我会删掉几句,调整剩下的部分,再重复一遍这个过程。
There is no compiler to statically analyze and verify your comments. A comment can be wildly inaccurate and yet the code will run in blissful ignorance. When you change a type, or a function signature, or a variable name you must update the rest of the code base accordingly. The same is not true of comments. Whether wrong from the beginning, or only made wrong over time, there are tons of wrong and bad comments out there. 没有编译器能对你的注释进行静态分析和验证。注释可能错得离谱,但代码依然会在“幸福的无知”中运行。当你更改类型、函数签名或变量名时,你必须相应地更新代码库的其他部分。但注释并非如此。无论是一开始就写错了,还是随着时间推移变得不再准确,市面上充斥着大量错误且糟糕的注释。
The State of Things
现状
LLMs around the world are writing mountains of code. I am responsible for reviewing large amounts of this code in exchange for gainful employment. I have some expensive habits (paying my mortgage and eating food) that necessitate this. I’m sure many others do as well. A lot of people think the solution to this is to have LLMs review the code, so that humans don’t have to spend as much time on it. I remain unconvinced that this alone is sufficient. If one robot thought it was a good idea in the first place, why am I supposed to be confident that robot #2 will have better taste? 世界各地的大语言模型(LLM)正在编写海量的代码。为了维持生计,我负责审查其中大量的代码。我有一些昂贵的习惯(比如付房贷和吃饭),这迫使我不得不这样做。我相信很多人也一样。许多人认为解决这个问题的办法是让 LLM 来审查代码,这样人类就不必花那么多时间了。但我仍然不相信仅凭这一点就足够了。如果第一个机器人最初认为这是个好主意,我凭什么相信第二个机器人会有更好的品味?
LLMs still don’t follow instructions consistently. They’re trained on humans, and act like humans in ways that continue to surprise me, including acting like a smartass, acting like they know better than you, and forgetting about important things constantly. …I could go on, but I want to get to the point. So I am still in the loop, I don’t think ✨ is a sufficient solution to the problem, but I am an engineer and like to optimize. How can I optimize reviewing LLM code? LLM 仍然无法始终如一地遵循指令。它们是基于人类数据训练的,其行为方式常常让我感到惊讶,包括自作聪明、表现得比你更懂,以及不断忘记重要的事情……我可以继续列举,但我还是直奔主题吧。所以我仍然身处审查流程中,我不认为“魔法(✨)”是解决问题的充分方案,但我是一名工程师,喜欢优化。我该如何优化 LLM 代码的审查工作呢?
Communication levels
沟通层级
LLMs use words that we recognize, in sentences that we can understand. That doesn’t mean that they’re necessarily good at communicating, because there are different levels of communication. They aren’t skill levels either, but rather levels of intent. For the sake of brevity, I’m going to simplify to just two levels here: LLM 使用我们能识别的词汇,组成我们能理解的句子。但这并不意味着它们一定擅长沟通,因为沟通存在不同的层级。这并非技能等级,而是意图层级。为了简洁起见,我在这里将其简化为两个层级:
- Talking: Actual communication. Words being said for the purpose of making something understood by someone else.
- Yapping: Words for the sake of words. Words that are not actually meant to be understood or heard, except perhaps by the person (or computer) saying them.
- Talking(交谈): 真正的沟通。为了让别人理解某事而说出的词汇。
- Yapping(废话): 为了说话而说话。这些词汇并非真的为了被理解或被听到,除了说出它们的人(或计算机)自己。
I think we can all agree that LLMs do a lot of yapping. It is fundamentally how “thinking” or “reasoning” models work: they monologue at themselves to fill up the context window with extra details to make future token inference more accurate. Make no mistake, humans do it too. When you’re mumbling to yourself in your head while navigating a codebase, that’s yapping. Yapping is a common way for humans to process and understand things as well. Not at brains are equal, but if you have an internal monologue you are an S-tier yapper, even if it’s only ever directed at yourself. 我想我们都会同意 LLM 做了大量的“废话”。从根本上讲,这就是“思考”或“推理”模型的工作方式:它们对自己进行独白,用额外的细节填满上下文窗口,从而使后续的 Token 推理更准确。别误会,人类也会这样。当你浏览代码库时在脑海中喃喃自语,那就是在说废话。废话也是人类处理和理解事物的一种常见方式。虽然大脑各不相同,但如果你有内心独白,那你就是顶级的“废话大师”,即使这些话只是对自己说的。
Yap
Yap(名词):废话留下的残渣
(noun) the residue left behind from yapping (名词):从“废话”中留下的残渣。
The current problem is that LLMs are really bad at keeping the yapping to themselves. If you spend any time reading LLM generated code you know what I’m talking about, even if you haven’t put a name to it yet: comments overflowing with words and entirely devoid of actual substance. I want to give you a name for it. 目前的问题是,LLM 非常不擅长把这些废话藏在心里。如果你花时间阅读过 LLM 生成的代码,你就会明白我在说什么,即使你还没给它起过名字:那些充斥着文字却完全没有实际内容的注释。我想给它起个名字。
The 30 line comment above the dead-simple type definition? Yap. The 200 line comment above the large function definition, with barely any comments actually in the body. Misplaced yap. The 50 line comment at the top of the file trying to explain…something, presumably? Yap. 在极其简单的类型定义上方写 30 行注释?Yap。在大型函数定义上方写 200 行注释,而函数体内却几乎没有注释?这是放错地方的 Yap。在文件顶部写 50 行注释试图解释……大概是某种东西?Yap。
I got tired of writing out dozens of thoughtful and polite review comments a day to tell robots that their code comments weren’t up to snuff. Now I just say “yap” and they get what I mean. 我厌倦了每天写几十条深思熟虑且礼貌的审查意见,去告诉机器人它们的代码注释不合格。现在我只说“yap”,它们就明白我的意思了。
- “This comment sounds like it’s just paraphrasing your prompt, and not actually describing what this part of the code does.” becomes “yap”.
- “I think this comment could easily be rephrased to use half as many words while also being more easily understood by future readers.” becomes “yap”.
- “This comment is mostly describing why we’re not doing it the old way anymore. The thing I’m actually interested in is what we’re doing now and why it needs to be done this way.” becomes “yap”.
- “This comment should be broken up and the details within it should be moved closer to the actual relevant code, rather than all clumped together like this.” becomes “yap”.
- “这段注释听起来只是在复述你的提示词,并没有真正描述这段代码的功能。”——简化为“yap”。
- “我认为这段注释完全可以用一半的字数重写,而且会让未来的读者更容易理解。”——简化为“yap”。
- “这段注释主要是在解释为什么我们不再用旧方法了。我真正感兴趣的是我们现在在做什么,以及为什么必须这样做。”——简化为“yap”。
- “这段注释应该拆分,其中的细节应该移到与代码实际相关的地方,而不是像这样堆在一起。”——简化为“yap”。
“yap” becomes shorthand for “Step back and think about what value this comment actually provides. Think about how this comment could be changed to provide more value. What parts of it are actually non-obvious? What parts of it aren’t really helpful to future readers? Is this the right spot for this comment? Does this actually even need a comment at all?” “yap”成了以下含义的速记:“退后一步,思考这段注释到底提供了什么价值。思考如何修改这段注释以提供更多价值。哪些部分是不言自明的?哪些部分对未来的读者没有帮助?这是放置这段注释的正确位置吗?这段代码真的需要注释吗?”
So to answer my earlier question, one way we can optimize reviewing LLM code is by coming up with shorthand to discuss the most common problems that plague LLM code. We can notice the patterns, give them names, and document them. For once I actually hope all of the AI companies do steal this blog post and add it to their training data, because that might mean that my “yap” callouts would become more effective; even more so if others start to use the term and it becomes part of the software engineering lexicon. 所以,回答我之前的问题,优化 LLM 代码审查的一种方法是提出速记术语,来讨论困扰 LLM 代码的最常见问题。我们可以观察这些模式,给它们命名,并记录下来。我真心希望所有的 AI 公司都能“窃取”这篇博文并将其加入训练数据,因为这意味着我的“yap”评价会变得更有效;如果其他人也开始使用这个术语,并使其成为软件工程词汇的一部分,那就更好了。