Introducing CUDA Rust: Two Tracks for Writing GPU Kernels

介绍 CUDA Rust：编写 GPU 内核的两条路径

Sep 08, 2026 | By Sri Koundinyan, Melih Elibol and Jonathan Bentz 2026年9月8日 | 作者：Sri Koundinyan, Melih Elibol 和 Jonathan Bentz

AI-Generated Summary

AI 生成摘要

cuda-oxide provides a custom rustc codegen backend that compiles SIMT-style GPU kernels written in Rust directly to PTX using the Pliron IR framework and LLVM. cuda-oxide 提供了一个自定义的 rustc 代码生成后端，利用 Pliron IR 框架和 LLVM 将用 Rust 编写的 SIMT 风格 GPU 内核直接编译为 PTX。

cutile-rs enables Tile-based GPU programming in stable Rust, where the compiler manages thread mapping and memory layout through CUDA Tile IR JIT compilation. cutile-rs 在稳定版 Rust 中实现了基于 Tile（分块）的 GPU 编程，编译器通过 CUDA Tile IR JIT 编译来管理线程映射和内存布局。

Both projects enforce memory safety at compile time: cuda-oxide uses DisjointSlice and launch contracts to prevent aliasing, while cutile-rs uses tensor partitioning and ownership to guarantee exclusive access. 这两个项目都在编译时强制执行内存安全：cuda-oxide 使用 DisjointSlice 和启动契约（launch contracts）来防止别名问题，而 cutile-rs 则通过张量分区和所有权机制来保证独占访问。

cuda-oxide requires a pinned nightly toolchain and LLVM, whereas cutile-rs runs on stable Rust 1.89+ with CUDA 13.3 and no custom LLVM. cuda-oxide 需要固定的 nightly 工具链和 LLVM，而 cutile-rs 可在稳定版 Rust 1.89+ 和 CUDA 13.3 环境下运行，无需自定义 LLVM。

cutile-rs is published on crates.io and already used in HuggingFace’s Grout inference engine and mistral.rs, while cuda-oxide remains in early alpha. cutile-rs 已发布在 crates.io 上，并已被用于 HuggingFace 的 Grout 推理引擎和 mistral.rs 中，而 cuda-oxide 目前仍处于早期 Alpha 阶段。

NVIDIA plans to support inter-language interoperability between CUDA Rust, CUDA C++, and CUDA Python so the choice of frontend does not lock developers out of other ecosystems. NVIDIA 计划支持 CUDA Rust、CUDA C++ 和 CUDA Python 之间的跨语言互操作性，确保开发者不会因选择某种前端而被锁定在特定的生态系统中。

In September 2026, NVIDIA announced it is leaning into native GPU programming in Rust. CUDA C++ and CUDA Python are mature, enterprise-grade toolchains, and NVIDIA will be growing and maturing CUDA Rust into 2027 and beyond. 2026 年 9 月，NVIDIA 宣布将大力投入 Rust 原生 GPU 编程。CUDA C++ 和 CUDA Python 是成熟的企业级工具链，NVIDIA 将在 2027 年及以后持续发展并完善 CUDA Rust。

The systems layer of AI spans inference engines, serving infrastructure, drivers, and agent runtimes, and it churns constantly as models and techniques change. More and more of it is written in Rust, which catches whole classes of bugs at compile time without giving up performance. NVIDIA is part of that shift for the same reason. The Nova Linux driver is written in Rust. NVIDIA Dynamo is built on a Rust core. NVTX has Rust bindings. AI 的系统层涵盖了推理引擎、服务基础设施、驱动程序和代理运行时，随着模型和技术的更迭，这一层在不断演变。越来越多的系统组件开始使用 Rust 编写，因为它能在不牺牲性能的前提下，在编译时捕获整类错误。NVIDIA 出于同样的原因参与了这一转变。Nova Linux 驱动程序是用 Rust 编写的，NVIDIA Dynamo 构建在 Rust 核心之上，NVTX 也拥有 Rust 绑定。

The GPU kernel is the exception. You can launch kernels from Rust, but the kernel itself often has to be written in another language. NVIDIA CUDA Rust closes that gap. GPU kernels can be written in Rust, compiled natively to PTX, rather than a wrapper around code from somewhere else. GPU 内核是个例外。虽然你可以从 Rust 中启动内核，但内核本身往往必须用其他语言编写。NVIDIA CUDA Rust 填补了这一空白。GPU 内核现在可以用 Rust 编写并原生编译为 PTX，而不再仅仅是其他代码的包装器。

There are two tracks to use Rust, matching the two tracks CUDA itself has. SIMT is the model you already write in CUDA C++ or numba-cuda. You indicate what one thread does, and launch thousands of them. Tile is a newer programming model, which is also available in C++ and Python. All of these frontends let you say what one tile of data does, and the Tile IR compiler does the rest. 使用 Rust 有两条路径，这与 CUDA 本身的两条路径相对应。SIMT 是你已经在 CUDA C++ 或 numba-cuda 中使用的模型：你定义单个线程的行为，然后启动数千个线程。Tile 是一种较新的编程模型，同样适用于 C++ 和 Python。所有这些前端都允许你定义一个数据块（Tile）的操作，剩下的工作由 Tile IR 编译器完成。

When you are picking one to build on, reach for Tile first. The compiler decides how tiles map onto each architecture, so your source doesn’t encode architecture-specific choices, and you drop to SIMT when you need that control or want to manage memory and threads yourself. 在选择构建路径时，建议优先考虑 Tile。编译器会决定 Tile 如何映射到不同的架构，因此你的源代码不会包含特定于架构的硬编码选择；当你需要精细控制或想要自行管理内存和线程时，再转向 SIMT。

Which language you reach for is a separate question from which model. Use the CUDA exposure that best fits the stack you already have. The two projects below are for when that stack is Rust. We plan to support inter-language interop, so the choice does not lock you out of the others. 选择哪种语言与选择哪种模型是两个独立的问题。请使用最适合你现有技术栈的 CUDA 方案。以下两个项目适用于你的技术栈为 Rust 的情况。我们计划支持跨语言互操作，因此你的选择不会将你限制在单一生态中。

The SIMT track: cuda-oxide

SIMT 路径：cuda-oxide

cuda-oxide is a custom rustc codegen backend. It intercepts compilation, routes #[kernel] functions through Rust MIR, the community Pliron IR framework, and LLVM IR down to PTX, and hands everything else to the standard backend. The GPU dialects on top of Pliron are ours. The dialects and every transform stay in Rust until the standard LLVM backend takes over. cuda-oxide 是一个自定义的 rustc 代码生成后端。它会拦截编译过程，将 #[kernel] 函数通过 Rust MIR、社区的 Pliron IR 框架和 LLVM IR 路由至 PTX，并将其他所有内容交给标准后端处理。Pliron 之上的 GPU 方言由我们开发。在标准 LLVM 后端接管之前，这些方言和所有转换过程都保留在 Rust 中。

You will need Linux, a GPU with compute capability 8.0 or later, a CUDA toolkit (12.x or newer), clang with its libclang headers, and the pinned nightly toolchain. cargo oxide doctor checks all of it, including the optional system LLVM. 你需要 Linux 系统、计算能力 8.0 或更高的 GPU、CUDA 工具包（12.x 或更新版本）、带有 libclang 头文件的 clang，以及固定的 nightly 工具链。cargo oxide doctor 可以检查所有这些依赖项，包括可选的系统 LLVM。

Install cargo-oxide, the Cargo subcommand that drives the build: 安装驱动构建过程的 Cargo 子命令 cargo-oxide：

cargo +nightly-2026-04-03 install --git https://github.com/NVlabs/cuda-oxide.git cargo-oxide

Then scaffold a project and run it. The template is a complete vector addition program: 然后构建项目并运行。该模板是一个完整的向量加法程序：

cargo oxide new vecadd_demo
cd vecadd_demo
cargo oxide doctor
cargo oxide run

The first cargo oxide run builds the codegen backend, so expect it to take a while. Later runs reuse the cache. It prints PASSED: all 1024 elements correct. 第一次运行 cargo oxide run 时会构建代码生成后端，因此需要一些时间。后续运行将重用缓存。程序会输出 PASSED: all 1024 elements correct。