Debunking "Faster than Rust": Malicious Benchmarks in Tov

本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

Title: Debunking “Faster than Rust”: Malicious Benchmarks in Tov

Tov is a new programming language marketed to be “[a] programming language for coding agents. It reads like TypeScript and builds native binaries that beat Rust’s.” Benchmarks in their Git repository show that they perform better than Rust, in some cases over 10× better. However, they also briefly mentioned the internals of Tov: Tov is a compiler. It checks the TypeScript your agent wrote, writes it out as C and hands that to clang, so what you ship is one native binary with nothing to install next to it.

Tov 是一门新兴的编程语言，其市场宣传称它是“专为编码代理设计的编程语言。它读起来像 TypeScript，并能构建出击败 Rust 的原生二进制文件。”其 Git 仓库中的基准测试显示，它们的性能优于 Rust，在某些情况下甚至快了 10 倍以上。然而，他们也简要提到了 Tov 的内部机制：Tov 是一个编译器。它检查代理编写的 TypeScript 代码，将其转换为 C 语言并交给 clang 处理，因此你发布的是一个无需额外安装任何东西的原生二进制文件。

A sane engineer would immediately doubt the correctness of their benchmarks. Rust has pretty much the same performance as C, and if Tov transpiles to C, saying Tov beats Rust is mostly equivalent to saying C beats C. That’s contradiction right there. Let’s ignore the stats for axum, maybe tokio has some overhead, like maybe; but clearly there’s no way you get 1.6× faster than Rust in strings. You have to be lying. That’s what a sane engineer would normally think.

任何理智的工程师都会立即怀疑其基准测试的准确性。Rust 的性能与 C 几乎相同，如果 Tov 是将代码转译为 C，那么说 Tov 击败 Rust 在很大程度上等同于说 C 击败了 C。这本身就是一个矛盾。让我们忽略 axum 的统计数据，也许 tokio 确实存在一些开销，这有可能；但显然，在字符串处理方面，你不可能比 Rust 快 1.6 倍。这一定是在撒谎。这就是理智的工程师通常会有的想法。

In this blog post, we check 3 of their “inflated” benchmarks: bigint_fib, dispatch, and map_insert. We show that these benchmarks are maliciously crafted to slow Rust down. We have also benchmarked the improved Rust programs with their Tov programs to show that Rust has equivalent or better performance than Tov.

在这篇博文中，我们检查了他们三个“注水”的基准测试：bigint_fib、dispatch 和 map_insert。我们展示了这些基准测试是被恶意设计用来拖慢 Rust 速度的。我们还使用改进后的 Rust 程序与他们的 Tov 程序进行了对比测试，以证明 Rust 具有与 Tov 相当甚至更好的性能。

Set-ups: cargo build —release was used to compile the Rust programs, and tov build (which defaults to -O3) was used to compile the Tov programs.

环境设置：使用 cargo build --release 编译 Rust 程序，使用 tov build（默认开启 -O3 优化）编译 Tov 程序。

bigint_fib: This was the Rust code used in Tov’s benchmark. Their benchmarking results showed that Rust used 501 ms · 4.5 MB, while Tov used 168 ms · 3.1 MB. However, we cannot reproduce their results on any of our setups.

bigint_fib：这是 Tov 基准测试中使用的 Rust 代码。他们的测试结果显示 Rust 耗时 501 毫秒，内存占用 4.5 MB，而 Tov 耗时 168 毫秒，内存占用 3.1 MB。然而，我们在任何测试环境中都无法复现他们的结果。