Karpathy’s Pelican

Andrej Karpathy @karpathy We’re starting to leave the territory where you’d test an LLM by e.g. “create an svg of pelican on a bicycle”. As one idea to generalize it, I was interested what Opus 5 would do if I gave it the first paragraph of the Lord of the Rings, a 1M token budget (~$10) and asked for three js render of it. Opus went off for ~2 hours and wrote 5500 lines of code that (procedurally) rendered the story. It’s kind of janky but fun.

我们正逐渐脱离那种通过“画一只骑自行车的鹈鹕的 SVG”来测试大模型的阶段。为了将其泛化，我很好奇如果给 Opus 5 提供《指环王》的第一段文字，并给予 100 万 token 的预算（约 10 美元），让它用 Three.js 将其渲染出来，会发生什么。Opus 花了大约 2 小时，编写了 5500 行代码，（程序化地）渲染出了这个故事。虽然有点粗糙，但很有趣。

But it’s a bit mindboggling that the LLM has to place and orchestrate various polygon assets in (x,y,z) coordinates and write code that animates it all, and that it even does anything at all. I also like this kind of examples because no one in their right mind would ever spend the time to write something this custom but LLMs have all the stamina and patience in the world, so it’s an example where we go from “no one would ever do this” to “sure, why not, it’s ~free”.

令人难以置信的是，大模型竟然需要在大脑中规划并协调各种多边形资产在 (x, y, z) 坐标系中的位置，并编写代码让这一切动起来，而且它居然真的做到了。我喜欢这类例子，因为正常人绝不会花时间去写这种高度定制化的东西，但大模型拥有无穷的精力和耐心。这正是一个从“没人会做这种事”到“当然可以，反正几乎免费”的典型案例。

There might be a lot more. But I’m excited about creating hyper custom worlds that you can imagine dropping players into, e.g. here to participate in the LoTR story as a spectator NPC, or one of the characters, or etc. Something like an ephemeral GTA of X on demand.

未来可能还会有更多这样的应用。我非常期待能够创造出高度定制化的世界，让玩家可以置身其中，比如作为旁观的 NPC 或某个角色参与到《指环王》的故事中。这就像是按需生成的、转瞬即逝的“某某主题 GTA”。

Last thought is that the domain of worlds/games exposes a weakness in LLMs: they can’t easily audit their work because they aren’t able to efficiently and natively perceive videos or play games within them. Here, Opus 5 had to very slowly and painstakingly take screenshots at different points, and it messed up a few times and created a bunch of jank. An example of raw capability (multimodal, gameplay) that I think is still quite lacking.

最后一点想法是，游戏/虚拟世界领域暴露了大模型的一个弱点：它们无法轻松地审查自己的工作，因为它们无法高效且原生化地感知视频或在游戏中进行交互。在这个例子中，Opus 5 必须非常缓慢且费力地在不同节点截屏，期间还出错了几次，导致了一些瑕疵。这说明在原始能力（多模态、游戏交互）方面，我认为大模型仍有很大欠缺。

Andrej Karpathy @karpathy More on the pelican on the bicycle test from @simonw: simonwillison.net/2025/Jun/6/six… I uploaded the source here so it’s playable in the browser, forkable etc. karpathy.ai/lotr-movie/ Look out for GTA Hobbiton dropping before GTA VI :)

关于“骑自行车的鹈鹕”测试，更多内容请参考 @simonw 的文章：simonwillison.net/2025/Jun/6/six… 我已经把源代码上传到了这里，大家可以在浏览器中运行、分叉（fork）等：karpathy.ai/lotr-movie/。期待《GTA：霍比特人》在《GTA VI》之前发布 :)