Auto-research with codex: How I achieved a 232x Faster Kernel
Auto-research with codex: How I achieved a 232x Faster Kernel
使用 Codex 进行自动研究：我是如何实现比基准快 232 倍的内核的
Auto-research with codex: How I achieved a 232x Faster Kernel over baseline with Codex in GPU Mode’s qr_v2 problem 08 Jul, 2026 在 GPU Mode 的 qr_v2 问题中，我如何利用 Codex 实现比基准快 232 倍的内核（2026 年 7 月 8 日）
Intro
引言
GPU Mode, in collab with Core Automation, recently hosted an auto-research themed contest. The problem statement was to implement batched square compact-Householder QR factorization aka QR decomposition. I placed 12th out of 183 participants, ending up with a 232x speedup over the baseline solution. This post is about how I got there. I will go through my approach, learnings, and bottlenecks I ran into during the contest. It was my first serious attempt at auto-research. Some people will call this “loop engineering”, and honestly that is fine too. GPU Mode 最近与 Core Automation 合作举办了一场以“自动研究”为主题的竞赛。题目要求实现分批方形紧凑型 Householder QR 分解（即 QR 分解）。我在 183 名参赛者中排名第 12 位，最终实现了比基准方案快 232 倍的加速。这篇文章将介绍我是如何做到这一点的。我将回顾我在比赛中的方法、心得以及遇到的瓶颈。这是我第一次认真尝试“自动研究”。有些人可能会称之为“循环工程”（loop engineering），老实说，这也没什么不好。
Note that you don’t need to go through the mathematics or the problem itself in detail to follow most of this blog post. I have focused on my approach while keeping the math and the problem itself secondary as most people who will read this won’t have participated in the contest. You can check out the full contest page here: Problem Link and Leaderboard. This contest was part of GPU Mode’s Linear Algebra Kernels in the Age of Research series. 请注意，阅读本文的大部分内容并不需要深入了解数学或问题本身。我将重点放在我的方法上，而将数学和问题本身作为次要内容，因为大多数读者并没有参加过这次比赛。你可以在这里查看完整的比赛页面：问题链接和排行榜。本次竞赛是 GPU Mode“研究时代的线性代数内核”系列活动的一部分。
Problem intro
问题介绍
We were given a batch of square FP32 CUDA matrices A with shape batch x n x n, and had to return the same compact Householder QR representation as torch.geqrf(A): an H matrix whose upper triangle is R and whose lower triangle stores Householder vectors, plus a tau vector of reflector coefficients. The checker rebuilt Q with torch.linalg.householder_product(H, tau), took R = triu(H), and verified: A≈QR, Q⊤Q≈I, Q⊤A≈R.
我们得到了一批形状为 batch x n x n 的方形 FP32 CUDA 矩阵 A，并需要返回与 torch.geqrf(A) 相同的紧凑型 Householder QR 表示：一个 H 矩阵（其上三角为 R，下三角存储 Householder 向量）以及一个反射系数 tau 向量。检查器使用 torch.linalg.householder_product(H, tau) 重建 Q，取 R = triu(H)，并验证：A≈QR, Q⊤Q≈I, Q⊤A≈R。
Among correct submissions, the leaderboard ranked runtime by geometric mean across shapes and conditioning cases. The important sizes were batched square matrices like 512 x 512, with larger 1024, 2048, and 4096 cases too. Low-bit FP16, FP8, or NVFP4 was allowed internally, but returned factors still had to satisfy FP32-style QR checks. 在所有正确的提交中，排行榜根据不同形状和条件下的几何平均运行时间进行排名。重点关注的尺寸是诸如 512 x 512 的分批方形矩阵，同时也包含 1024、2048 和 4096 等更大尺寸的情况。内部允许使用低位宽的 FP16、FP8 或 NVFP4，但返回的因子仍必须满足 FP32 风格的 QR 检查。
Why this problem is auto-research-able
为什么这个问题适合自动研究
GPU Mode provides participants with the popcorn CLI making it agent-friendly. Agents can use this to test, benchmark, and submit to the leaderboard directly. The checker also provided shape-wise feedback along with the overall geometric mean timing. Astute observers will notice this is an apt setup for writing a loop. Agents yearn for tight feedback loops. They allow them to hill-climb to their heart’s content. GPU Mode 为参赛者提供了 popcorn CLI，使其对 AI Agent 非常友好。Agent 可以利用它直接进行测试、基准评估并提交到排行榜。检查器还提供了针对不同形状的反馈以及整体几何平均时间。敏锐的观察者会发现，这非常适合编写循环程序。Agent 渴望紧密的反馈循环，这使它们能够尽情地进行爬山算法优化。
GPU Mode contests usually give you some way to iterate on kernels. Either you submit directly, or a sponsor like Modal chips in credits. Here the organizers basically allowed unlimited submissions as long as you spaced them out. If you didn’t, the queues got long and everybody’s runs timed out. At one point the workspace even ran out of Modal credits because everyone had been hammering submissions. It’s a nice way to make learning accessible. Over the course of 14 days, I made over 1500 submissions. GPU Mode 的比赛通常会提供某种迭代内核的方法。你可以直接提交，或者由像 Modal 这样的赞助商提供算力额度。在这里，组织者基本上允许无限次提交，只要你控制好提交间隔即可。如果不这样做，队列会变长，导致所有人的运行超时。有一次，工作空间甚至因为大家疯狂提交而耗尽了 Modal 额度。这是一种让学习变得触手可及的好方法。在 14 天的时间里，我提交了超过 1500 次。
Learning Enough to Ask Better Questions
学习到足以提出更好的问题
I have known the basics of GPU kernel optimization (mainly in Triton with some understanding of CUDA) for a year, but haven’t worked in this domain professionally. What I am trying to tell you is that I was an underdog among the people around me on the leaderboard. The person just above me on the leaderboard (CUDA Colonel) is a principal engineer at NVIDIA. 我了解 GPU 内核优化的基础知识（主要是 Triton，对 CUDA 有一定了解）已经一年了，但并没有在该领域从事专业工作。我想说的是，在排行榜上我周围的人中，我处于劣势。排行榜上排在我前面的人（CUDA Colonel）是 NVIDIA 的首席工程师。
Anyway aura farming aside, since I know the basics and had recently read about GatedDeltaNet, I was fresh on the general GPU kernel lingo. The better you know something, the better you can prompt the LLMs, because you convert unknown unknowns into known unknowns. At the same time, it’s worth noting that this contest was doable without domain knowledge - like you probably won’t make it to the top 10, but you can get a respectable speedup over baseline by just relying on your harness/agent loop or whatever. 撇开这些不谈，由于我了解基础知识，并且最近读过关于 GatedDeltaNet 的内容，我对通用的 GPU 内核术语非常熟悉。你对某件事了解得越深，就越能更好地提示 LLM，因为你将“未知的未知”转化为了“已知的未知”。同时值得注意的是，即使没有领域知识，这次比赛也是可以参与的——虽然你可能进不了前 10 名，但仅靠你的工具链/Agent 循环，你也能获得比基准方案可观的加速。
My first steps in the contest were to learn what QR decomposition is and how it can be done. There are a bunch of ways to do it - like Gram-Schmidt and Householder reflections. The contest mandated Householder reflections. I went back and forth with Claude and watched a few YouTube videos to build intuition. After my discussions with Claude, it was clear that we needed to use the blocked Householder algorithm as the main architecture with the trailing WY-update. As it turns out, GPT-5.5 also had a good idea about this. 我在比赛中的第一步是学习什么是 QR 分解以及如何实现它。实现方法有很多种，比如 Gram-Schmidt 和 Householder 反射。比赛强制要求使用 Householder 反射。我与 Claude 反复交流，并观看了一些 YouTube 视频来建立直觉。在与 Claude 讨论后，很明显我们需要使用分块 Householder 算法作为主要架构，并配合尾随的 WY 更新。事实证明，GPT-5.5 对此也有很好的见解。