kornia / kornia
本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。
English | 简体中文 Docs • Try it Now • Tutorials • Examples • Blog • Community Kornia is a differentiable computer vision library that provides a rich set of differentiable image processing and geometric vision algorithms. Built on top of PyTorch, Kornia integrates seamlessly into existing AI workflows, allowing you to leverage powerful batch transformations, auto-differentiation and GPU acceleration. Whether you’re working on image transformations, augmentations, or AI-driven image processing, Kornia equips you with the tools you need to bring your ideas to life.
Kornia 是一个可微计算机视觉库，提供了一套丰富的可微图像处理和几何视觉算法。Kornia 基于 PyTorch 构建，可无缝集成到现有的 AI 工作流程中，让您能够利用强大的批处理变换、自动微分和 GPU 加速。无论您是在进行图像变换、增强还是 AI 驱动的图像处理，Kornia 都能为您提供实现想法所需的工具。
📢 Direction: Kornia is becoming the reference implementation and executable specification for differentiable computer vision and geometry in the PyTorch ecosystem — explicit conventions, conformance tests, and honest benchmarks over API growth. Read the Roadmap. When to use Kornia Kornia pays off where a hand-written version gets long or fragile, or where the alternatives leave the GPU: Breadth. 1,200+ public operators, most with no torchvision equivalent: camera models and epipolar geometry, minimal solvers and RANSAC, feature detection, description and matching, edge detection, morphology, Lab and Bayer color, image losses and metrics. torchvision overlaps mainly on resizing, cropping, flipping, color jitter and blur.
📢 方向：Kornia 正在成为 PyTorch 生态系统中可微计算机视觉和几何学的参考实现和可执行规范——相比于 API 的增长，我们更看重明确的约定、一致性测试和真实的基准测试。阅读路线图。何时使用 Kornia：当手写版本变得冗长或脆弱，或者替代方案需要离开 GPU 时，Kornia 的价值就体现出来了：广度。拥有 1200 多个公共算子，其中大多数在 torchvision 中没有对应项：相机模型和对极几何、最小求解器和 RANSAC、特征检测、描述和匹配、边缘检测、形态学、Lab 和 Bayer 颜色、图像损失和指标。torchvision 的重叠部分主要在于调整大小、裁剪、翻转、颜色抖动和模糊。
Per-sample parameters in one batched call. gaussian_blur2d(x, (9, 9), sigma) takes a (B, 2) sigma; rotate takes a (B,) angle, warp_affine / warp_perspective take (B, 2, 3) / (B, 3, 3) matrices and adjust_brightness a (B,) factor. torchvision’s functional ops apply one parameter per call. From Kornia 0.9.1, a sigma tensor and the enhancement value checks (gamma, contrast, solarize) are validated on the device, so a CUDA call, or an MPS call on PyTorch ≥ 2.13, needs no host synchronization for them.
在一次批处理调用中实现每个样本的参数化。例如，gaussian_blur2d(x, (9, 9), sigma) 接受 (B, 2) 的 sigma；rotate 接受 (B,) 的角度，warp_affine / warp_perspective 接受 (B, 2, 3) / (B, 3, 3) 的矩阵，adjust_brightness 接受 (B,) 的因子。torchvision 的函数式算子每次调用只能应用一个参数。从 Kornia 0.9.1 开始，sigma 张量和增强值检查（伽马、对比度、曝光）在设备上进行验证，因此 CUDA 调用或 PyTorch ≥ 2.13 上的 MPS 调用无需进行主机同步。
Batched minimal solvers and robust estimation. find_fundamental(x1, x2, method=“7POINT”) solves (B, 7, 2) samples and returns all real solutions in (B, 3, 3, 3), with zero matrices in the unused slots; the 8-point, 5-point essential (find_essential), DLT homography and PnP (solve_pnp_dlt) solvers are there too. RANSAC (homography, fundamental and essential models) samples and scores hypotheses on the device; by default its refinement and final selection run on the host in float64, and compile=True (PyTorch ≥ 2.14, CPU and CUDA) runs the whole estimation as one compiled graph on the device.
批处理最小求解器和鲁棒估计。find_fundamental(x1, x2, method=“7POINT”) 求解 (B, 7, 2) 个样本，并以 (B, 3, 3, 3) 的形式返回所有实数解，未使用的槽位填充零矩阵；8 点法、5 点本质矩阵 (find_essential)、DLT 单应性矩阵和 PnP (solve_pnp_dlt) 求解器也包含在内。RANSAC（单应性、基础矩阵和本质矩阵模型）在设备上对假设进行采样和评分；默认情况下，其细化和最终选择在主机上以 float64 运行，而 compile=True（PyTorch ≥ 2.14，CPU 和 CUDA）则将整个估计过程作为单个编译图在设备上运行。
Differentiable geometry. Depth to 3D and back (depth_to_3d_v2, unproject_points, project_points), warp_frame_depth, remap, camera models with Brown–Conrady and Kannala–Brandt distortion, and Lie groups (So3, Se3) keep gradients with respect to depth, pose and camera parameters. Annotated augmentation. AugmentationSequential applies one sampled transform to images, masks, boxes and keypoints together and can invert it, in 2D; the 3D augmentations transform volumes, without annotations or an inverse. On CUDA, augmentations move their sampled parameters to the GPU without blocking it, and a sync-guard test keeps every public augmentation free of avoidable host synchronizations.
可微几何。深度转 3D 及反向转换 (depth_to_3d_v2, unproject_points, project_points)、warp_frame_depth、重映射、带有 Brown–Conrady 和 Kannala–Brandt 畸变的相机模型，以及李群 (So3, Se3) 均保留了关于深度、位姿和相机参数的梯度。带标注的增强。AugmentationSequential 可将一次采样的变换同时应用于图像、掩码、边界框和关键点，并支持 2D 反转；3D 增强则变换体数据，不带标注或反转功能。在 CUDA 上，增强操作将采样的参数移动到 GPU 而不会阻塞它，同步保护测试确保每个公共增强操作都避免了不必要的主机同步。
Local features on the GPU. SIFT and HardNet, DISK, ALIKED, XFeat, DeDoDe, LoFTR and LightGlue, with mutual-nearest-neighbour and ratio-test matchers. Morphology with arbitrary structuring elements on batched tensors. Compiles and exports. Nearly all probed operator configurations run under torch.compile and match eager, and over 1,000 export to ONNX; the result for every operator is on the export support page.
GPU 上的局部特征。包括 SIFT、HardNet、DISK、ALIKED、XFeat、DeDoDe、LoFTR 和 LightGlue，并配有互近邻和比率测试匹配器。支持在批处理张量上使用任意结构元素的形态学操作。编译与导出。几乎所有测试过的算子配置都能在 torch.compile 下运行并与即时模式匹配，且超过 1000 个算子可导出为 ONNX；每个算子的结果均可在导出支持页面查看。
Fast where it is designed to be: batched tensors on a GPU. In the published 0.9.0 flagship benchmarks (RTX 4090, batch 32), many filters, warps, feature-matching steps and image metrics run about 10× to 70× faster in eager mode than per-image OpenCV or scikit-image on the CPU, and torch.compile takes several of them past 100×. On the small overlap with torchvision on the same GPU, eager Kornia is slower; compiled Kornia is faster for hue, color jitter and Gaussian blur and still slower for flips, crops and brightness. OpenCV wins single-image uint8 CPU work. Results vary with hardware and workload; every number, win and loss is published on the performance page.
在设计目标上表现迅速：GPU 上的批处理张量。在已发布的 0.9.0 旗舰基准测试（RTX 4090，批大小 32）中，许多滤波器、变换、特征匹配步骤和图像指标在即时模式下比 CPU 上的单图像 OpenCV 或 scikit-image 快约 10 到 70 倍，而 torch.compile 则使其中一些超过了 100 倍。在与 torchvision 重叠的小部分功能上，即时模式下的 Kornia 在同一 GPU 上较慢；编译后的 Kornia 在色调、颜色抖动和高斯模糊方面更快，但在翻转、裁剪和亮度方面仍较慢。OpenCV 在单图像 uint8 CPU 任务中胜出。结果因硬件和工作负载而异；每一个数据、胜负情况都发布在性能页面上。
These operators run on the input’s device with no NumPy round trip, and most are differentiable; robust estimation (RANSAC) and the extraction methods of learned features such as DeDoDe and XFeat run without gradients. For a single fixed-parameter operation, such as one blur or one resize, a few lines of PyTorch or torchvision are just as good. Key Components Differentiable Image Processing Kornia provides a comprehensive suite of image processing operators, all differentiable and ready to integrate into deep learning pipelines. Filters: Gaussian, Sobel, Median, Box Blur, etc. Transformations: Affine, Homography, Perspective, etc. Enhancements: Histogram Equalization, CLAHE, Gamma Correction, etc. Edge Detection: Canny, Laplacian, Sobel, etc. … check our docs for more.
这些算子在输入所在的设备上运行，无需 NumPy 的往返转换，且大多数是可微的；鲁棒估计 (RANSAC) 以及 DeDoDe 和 XFeat 等学习特征的提取方法在运行时不计算梯度。对于单个固定参数的操作（如一次模糊或一次调整大小），几行 PyTorch 或 torchvision 代码同样有效。关键组件：可微图像处理。Kornia 提供了一套全面的图像处理算子，全部可微并可随时集成到深度学习流水线中。滤波器：高斯、Sobel、中值、盒状模糊等。变换：仿射、单应性、透视等。增强：直方图均衡化、CLAHE、伽马校正等。边缘检测：Canny、拉普拉斯、Sobel 等……查看我们的文档了解更多。
Advanced Augmentations Perform powerful data augmentation with Kornia’s built-in functions, ideal for training AI models with complex augmentation pipelines. Augmentation Pipeline: AugmentationSequential, PatchSequential, VideoSequential, etc. Automatic Augmentation: AutoAugment, RandAugment, TrivialAugment. AI Models Leverage pre-trained AI models optimized for a variety of vision tasks, all within the Kornia ecosystem. Face Detection: YuNet Feature Matching: LoFTR, LightGlue Feature Descriptor: DISK, DeDoDe, SOLD2 Segmentation: SAM Classification: MobileViT, VisionTransformer. See here for some of the methods that we support! (>500 ops in total !)
高级增强。使用 Kornia 的内置函数执行强大的数据增强，非常适合训练具有复杂增强流水线的 AI 模型。增强流水线：AugmentationSequential、PatchSequential、VideoSequential 等。自动增强：AutoAugment、RandAugment、TrivialAugment。AI 模型：在 Kornia 生态系统内利用针对各种视觉任务优化的预训练 AI 模型。人脸检测：YuNet；特征匹配：LoFTR、LightGlue；特征描述符：DISK、DeDoDe、SOLD2；分割：SAM；分类：MobileViT、VisionTransformer。点击此处查看我们支持的部分方法！（总计超过 500 个算子！）
Category Methods/Models Image Processing - Color conversions (RGB, Grayscale, HSV, etc.)- Geometric transformations (Affine, Homography, Resizing, etc.)- Filtering (Gaussian blur, Median blur, etc.)- Edge detection (Sobel, Canny, etc.)- Morphological operations (Erosion, Dilation, etc.) Augmentation - Random cropping, Erasing - Random geometric transformations (Affine, flipping, Fish Eye, P
类别 方法/模型 图像处理 - 颜色转换（RGB、灰度、HSV 等）- 几何变换（仿射、单应性、调整大小等）- 滤波（高斯模糊、中值模糊等）- 边缘检测（Sobel、Canny 等）- 形态学操作（腐蚀、膨胀等） 增强 - 随机裁剪、擦除 - 随机几何变换（仿射、翻转、鱼眼、P