We measured how much time our team wasted on flaky tests. The numbers were ugly

我们统计了团队在“不稳定测试”上浪费的时间，结果触目惊心

Like most teams using GitHub Actions, we’d gotten used to the ritual: push code, wait for CI, see a red build, re-run it, hope it passes this time. “It’s probably flaky” became the default response to any test failure — including real ones. 像大多数使用 GitHub Actions 的团队一样，我们已经习惯了这种例行公事：推送代码，等待 CI，看到构建失败（红灯），重新运行，祈祷这次能通过。“大概是不稳定测试（flaky test）吧”成了应对任何测试失败的默认借口——甚至包括那些真正的错误。

We decided to actually measure the damage. Over 30 days on a single repo: 我们决定量化这些损失。在 30 天内，针对单个代码仓库的统计结果如下：

• 842 CI runs → 117 failures (13.9% failure rate)
• 842 次 CI 运行 → 117 次失败（失败率 13.9%）
• 31.5 developer hours spent investigating and re-running
• 耗费 31.5 个开发者工时用于排查和重新运行
• $426 in CI compute burned on re-runs that shouldn’t have been needed
• 浪费了 426 美元的 CI 计算资源在不必要的重试上
• 1 regression shipped to production because a real failure was dismissed as “just flaky”
• 1 次回归错误被发布到生产环境，因为真正的失败被误认为是“不稳定测试”而遭到忽视

The worst part? Nobody could tell us which tests were flaky. We had a vague sense — “that login test is weird” — but no actual inventory. And without an inventory, you can’t fix what you can’t see. 最糟糕的是什么？没人能说清楚到底是哪些测试不稳定。我们只有一种模糊的感觉——“那个登录测试怪怪的”——但没有实际的清单。而没有清单，你就无法修复那些看不见的问题。

So we built Retestees — a tool that connects to your GitHub Actions in 2 minutes and gives you a CI Waste Report showing: 因此，我们开发了 Retestees——一个可以在 2 分钟内连接到你的 GitHub Actions 并为你提供 CI 浪费报告的工具，报告内容包括：

• Which tests fail repeatedly without code changes (true flaky tests)
• 哪些测试在没有代码变更的情况下反复失败（真正的“不稳定测试”）
• How much developer time and CI cost each one wastes
• 每个测试浪费了多少开发者时间和 CI 成本
• Which workflows are the least stable, ranked by failure rate
• 哪些工作流最不稳定（按失败率排序）
• A clear priority list so you fix the most expensive flaky tests first
• 清晰的优先级列表，让你优先修复代价最高的那些不稳定测试

No code changes. No config files. No test framework plugins. You connect your repo, and we analyze your existing CI history. 无需修改代码，无需配置文件，无需测试框架插件。你只需连接你的仓库，我们就会分析你现有的 CI 历史记录。

We’re looking for teams to try it free. If you’re on GitHub Actions and tired of re-running builds, we’ll generate a free CI Waste Report for your repo — no credit card, no commitment. You’ll see exactly where your CI time and money are going. 我们正在寻找愿意免费试用的团队。如果你正在使用 GitHub Actions 并且厌倦了反复重试构建，我们将为你的仓库生成一份免费的 CI 浪费报告——无需信用卡，无需任何承诺。你将清楚地看到你的 CI 时间和金钱到底花在了哪里。

Get your free CI Waste Report (We also have a $29/mo beta plan if you want ongoing monitoring and alerts. Details on the site.) 获取你的免费 CI 浪费报告（如果你需要持续的监控和警报，我们还提供每月 29 美元的测试版计划。详情请见网站。）

Would love feedback from anyone dealing with this. What’s your worst flaky test story? 非常欢迎正在处理此类问题的开发者提供反馈。你遇到过最糟糕的“不稳定测试”故事是什么？