Introducing @huggingface/kernels: 200+ WebGPU Kernels for Local AI

Introducing @huggingface/kernels: 200+ WebGPU Kernels for Local AI 介绍 @huggingface/kernels：为本地 AI 提供 200 多个 WebGPU 内核

One of our biggest goals on the WebAI team at Hugging Face is to make browser inference as fast and as user-friendly as possible. Getting there is a multi-layer effort: models need browser-friendly representations, runtimes need to build efficient execution plans, and the individual GPU operations at the bottom of the stack need to make the most of many different devices and browser implementations. Hugging Face WebAI 团队最大的目标之一，就是让浏览器端的推理尽可能快速且易于使用。实现这一目标需要多层面的努力：模型需要适配浏览器的表示形式，运行时需要构建高效的执行计划，而位于技术栈底层的各个 GPU 操作则需要充分利用各种不同的设备和浏览器实现。

Today, we are releasing the first layer of that effort: @huggingface/kernels, a minimal library for loading and running optimized WebGPU kernels from the Hugging Face Hub, together with an initial collection of 207 kernels at huggingface.co/webgpu-kernels. The collection covers operations used across a wide variety of machine learning architectures and workloads. 今天，我们发布了这一努力的第一层成果：@huggingface/kernels。这是一个轻量级库，用于从 Hugging Face Hub 加载并运行经过优化的 WebGPU 内核，同时我们还在 huggingface.co/webgpu-kernels 上发布了首批 207 个内核。该集合涵盖了各种机器学习架构和工作负载中使用的操作。

More importantly, each kernel is published as a complete, versioned package: its interface, shader templates, correctness cases, benchmark cases, and usage instructions all live together on the Hub. We are also launching Fleet, an in-browser GPU benchmarking and testing suite that runs and scores the kernels on your hardware. 更重要的是，每个内核都作为完整的版本化包发布：其接口、着色器模板、正确性测试用例、基准测试用例和使用说明都统一托管在 Hub 上。我们还推出了 Fleet，这是一个浏览器内的 GPU 基准测试和评估套件，可以在您的硬件上运行并为内核评分。

Beyond the results for your own machine, Fleet gives the community a way to contribute performance and correctness evidence from devices we could never cover in a conventional test lab. With your consent, every run adds private evidence that can help us find failures (incorrect results, pathologically slow cases, etc.), improve kernel variants, and make better optimization decisions across real-world hardware. 除了为您提供本机测试结果外，Fleet 还为社区提供了一种贡献性能和正确性数据的方式，这些数据来自我们在传统测试实验室中无法覆盖的各种设备。在您同意的情况下，每次运行都会添加私有证据，帮助我们发现故障（如结果错误、异常缓慢的情况等）、改进内核变体，并在真实硬件环境下做出更好的优化决策。

TL;DR

简而言之

207 WebGPU kernels, published as individual repositories in the webgpu-kernels organization. Apache-2.0 licensed. 207 个 WebGPU 内核，以独立仓库的形式发布在 webgpu-kernels 组织下，采用 Apache-2.0 许可证。
A JavaScript loader, @huggingface/kernels, which downloads, prepares, and runs kernels directly from the Hub. 一个 JavaScript 加载器 @huggingface/kernels，可直接从 Hub 下载、准备并运行内核。
Explicit contracts and reproducible evidence for every kernel, including manifests, correctness tests, benchmark cases, and WGSL shader templates. 每个内核都有明确的契约和可复现的证据，包括清单、正确性测试、基准测试用例和 WGSL 着色器模板。
Fleet, a browser-based benchmarking tool that crowdsources correctness and performance evidence across real-world GPUs to help us improve kernels and their variants. Fleet，一个基于浏览器的基准测试工具，通过众包方式收集真实 GPU 上的正确性和性能证据，以帮助我们改进内核及其变体。

Why start with kernels?

为什么要从内核开始？

A model running in the browser eventually becomes a sequence of GPU operations: matrix multiplications, normalizations, convolutions, attention primitives, quantization operations, data-layout transformations, and many more. WebGPU makes these operations available across modern browsers through a portable API, while WGSL provides a common language for the shaders that execute them. 在浏览器中运行的模型最终会变成一系列 GPU 操作：矩阵乘法、归一化、卷积、注意力原语、量化操作、数据布局转换等等。WebGPU 通过可移植的 API 使这些操作能够在现代浏览器中运行，而 WGSL 则为执行这些操作的着色器提供了通用语言。

Portability, however, does not automatically mean performance. Two shaders can implement the same operation and produce the same output while behaving completely differently across different accelerators. Workgroup sizes, memory access patterns, vectorization, data types, and fusion strategies can all affect performance. The best choice can also change with the input shape, device, browser, and available WebGPU features. 然而，可移植性并不自动等同于高性能。两个着色器可以实现相同的操作并产生相同的输出，但在不同的加速器上表现却截然不同。工作组大小、内存访问模式、向量化、数据类型和融合策略都会影响性能。最佳选择还会随着输入形状、设备、浏览器和可用的 WebGPU 功能而变化。

This is why kernels form a foundational layer of fast browser inference. Higher-level runtimes can only be as efficient as the operations they dispatch. By making those operations individually discoverable, testable, benchmarkable, and versioned, we can improve the foundation independently while keeping a stable contract for the layers above it. 这就是为什么内核构成了快速浏览器推理的基础层。高级运行时只有在它们调用的操作足够高效时才能发挥作用。通过使这些操作能够被单独发现、测试、基准评估和版本化，我们可以在保持上层稳定契约的同时，独立地改进底层基础。

A kernel repository, not just a shader

不仅仅是着色器，而是内核仓库

Each kernel in the collection has its own repository and kernel card. The card documents the operation’s semantics, inputs, outputs, attributes, supported data types, source files, and a ready-to-run @huggingface/kernels example. 集合中的每个内核都有自己的仓库和内核卡片。卡片记录了操作的语义、输入、输出、属性、支持的数据类型、源文件以及一个可直接运行的 @huggingface/kernels 示例。

For example, ai.onnx.Add implements elementwise addition with multidirectional broadcasting. It is one of the simplest operations in a neural network, used everywhere from residual connections to adding a bias. Its card documents the two inputs, the broadcasted output shape, supported data types, and the variants available for different shapes and devices. 例如，ai.onnx.Add 实现了带有多向广播的逐元素加法。它是神经网络中最简单的操作之一，从残差连接到添加偏置，随处可见。其卡片记录了两个输入、广播后的输出形状、支持的数据类型以及针对不同形状和设备可用的变体。

The ai.onnx.Add repository packages its manifest, correctness and benchmark cases, and WGSL shader templates together. Behind the card, the repository contains the artifacts needed to understand and evaluate the implementation: ai.onnx.Add 仓库将其清单、正确性和基准测试用例以及 WGSL 着色器模板打包在一起。在卡片背后，仓库包含了理解和评估该实现所需的工件：

manifest.json is the source of truth for the operation contract. It defines inputs, outputs, attributes, type constraints, and shape derivation rules. manifest.json 是操作契约的唯一事实来源。它定义了输入、输出、属性、类型约束和形状推导规则。
metadata.json records the kernel identifier, digests, and provenance. metadata.json 记录了内核标识符、摘要和来源。
test.json contains correctness cases, so an implementation can be checked against expected behavior. test.json 包含正确性测试用例，以便根据预期行为检查实现。
bench.json contains benchmark and tuning cases that represent the workloads used to evaluate the kernel. bench.json 包含用于评估内核的基准测试和调优工作负载用例。
*.wgsl.jinja files contain the parameterized WGSL implementations used to produce shaders for a particular request and device. *.wgsl.jinja 文件包含参数化的 WGSL 实现，用于为特定请求和设备生成着色器。

This structure turns a shader into a reusable software artifact. The interface is inspectable without reading WGSL, correctness and performance cases travel with the implementation, and published versions can be loaded explicitly rather than depending on an unversioned file URL. Our kernels can also serve as reference implementations for developers building custom WebGPU kernels or integrating these operations into their own runtimes. 这种结构将着色器转变为可重用的软件工件。无需阅读 WGSL 即可检查接口，正确性和性能测试用例随实现一同分发，且发布的版本可以被显式加载，而不是依赖于未版本化的文件 URL。我们的内核还可以作为参考实现，供开发人员构建自定义 WebGPU 内核或将这些操作集成到他们自己的运行时中。

Loading a kernel from the Hub

从 Hub 加载内核

Install the package from npm: 从 npm 安装包： npm install @huggingface/kernels@preview

Running these kernels requires a browser with WebGPU support. WebGPU availability depends on the browser, operating system, GPU, and driver. You can check for it in JavaScript with "gpu" in navigator. 运行这些内核需要支持 WebGPU 的浏览器。WebGPU 的可用性取决于浏览器、操作系统、GPU 和驱动程序。您可以在 JavaScript 中通过 "gpu" in navigator 进行检查。

@huggingface/kernels provides the bridge between a kernel repository and your application. Call getKernel with a Hub repository ID and a contract version, then invoke the returned function with typed input data and tensor shapes. Here is a small bias-add example: @huggingface/kernels 提供了内核仓库与您的应用程序之间的桥梁。使用 Hub 仓库 ID 和契约版本调用 getKernel，然后使用类型化的输入数据和张量形状调用返回的函数。以下是一个简单的偏置加法示例：

import { getKernel } from "@huggingface/kernels";

const add = await getKernel("webgpu-kernels/ai.onnx.Add", { version: 1 });
const { c } = await add({ a: ... });