Mastering Low-Precision AI: FP8 and FP4 Support Across Frameworks in Mid-2026

掌握低精度 AI：2026 年中各框架对 FP8 和 FP4 的支持情况

In mid-2026, FP8 and FP4 have become essential tools for making large-scale AI training and inference more efficient. FP8 uses two main formats—E4M3 for better precision on activations and weights, and E5M2 for wider dynamic range on gradients—while NVIDIA’s NVFP4 takes things further with 4-bit values and micro-block scaling (shared FP8 scales per 16 elements plus a tensor-level scale). 2026 年中，FP8 和 FP4 已成为提升大规模 AI 训练与推理效率的核心工具。FP8 主要使用两种格式：用于提升激活值和权重精度的 E4M3，以及用于提供更广梯度动态范围的 E5M2。而 NVIDIA 的 NVFP4 则更进一步，采用 4 位数值和微块缩放（Micro-block scaling，即每 16 个元素共享一个 FP8 缩放因子，外加一个张量级缩放因子）。

These formats dramatically cut memory use and increase throughput on modern GPUs compared with traditional BF16 or FP16, making it possible to train and serve bigger models on the same hardware. The benefits are clear: roughly 2× memory savings with FP8 and up to 3.5× with NVFP4, higher Tensor Core performance, and better energy efficiency. 与传统的 BF16 或 FP16 相比，这些格式显著降低了现代 GPU 的内存占用并提高了吞吐量，使得在相同硬件上训练和部署更大规模的模型成为可能。其优势显而易见：FP8 可节省约 2 倍内存，NVFP4 最高可节省 3.5 倍，同时还带来了更高的 Tensor Core 性能和更好的能效。

The trade-offs come from reduced numerical range and precision, which can lead to accuracy loss or instability unless carefully managed with techniques such as delayed scaling, stochastic rounding, Hadamard transforms, and selective quantization that skips sensitive layers. When these methods are applied properly, accuracy often stays within 1–2% of higher-precision baselines on real workloads. 其代价在于数值范围和精度的降低，如果不通过延迟缩放（delayed scaling）、随机舍入（stochastic rounding）、哈达玛变换（Hadamard transforms）以及跳过敏感层的选择性量化等技术进行精细管理，可能会导致精度损失或训练不稳定。当这些方法应用得当时，在实际工作负载中，精度通常能保持在高精度基准的 1–2% 以内。

Research has moved quickly from the foundational 2022 FP8 paper to 2025 studies showing stable FP4 pre-training of multi-billion-parameter models. Hardware support is mature for FP8 on Hopper GPUs and reaches its peak on Blackwell with native NVFP4 and MXFP8 acceleration. 研究进展迅速，从 2022 年奠基性的 FP8 论文，到 2025 年的研究已证明了数十亿参数模型在 FP4 下进行预训练的稳定性。硬件方面，Hopper GPU 对 FP8 的支持已趋于成熟，而在 Blackwell 架构上，通过原生的 NVFP4 和 MXFP8 加速，硬件支持达到了巅峰。

Among frameworks, PyTorch currently leads with native float8 dtypes, Transformer Engine for production training, and TorchAO for optimized inference. JAX offers solid support through Transformer Engine, TensorFlow/Keras provides simpler quantize-to-FP8 options but relies more on TensorRT for high performance, and libraries such as bitsandbytes remain useful for complementary 4-bit memory savings. 在各框架中，PyTorch 目前处于领先地位，提供原生的 float8 数据类型、用于生产训练的 Transformer Engine 以及用于优化推理的 TorchAO。JAX 通过 Transformer Engine 提供了稳健的支持；TensorFlow/Keras 提供了更简单的 FP8 量化选项，但在高性能场景下更多依赖 TensorRT；而 bitsandbytes 等库在实现 4 位内存节省方面依然非常实用。

Practical adoption is already strong for both training and inference, especially when teams start with proven recipes, monitor scaling factors, and prototype on smaller models. Workarounds for remaining gaps include casting unsupported operations to higher precision or using selective quantization. Overall, FP8 is now production-ready for most teams, while NVFP4 is rapidly becoming practical on Blackwell hardware for maximum efficiency. 目前，FP8 和 FP4 在训练和推理中的实际应用已经非常广泛，特别是当团队采用成熟的方案、监控缩放因子并在较小模型上进行原型设计时。针对剩余的兼容性缺口，可以通过将不支持的操作转换为高精度，或使用选择性量化来解决。总的来说，FP8 对大多数团队而言已具备生产就绪能力，而 NVFP4 正在 Blackwell 硬件上迅速普及，以实现极致效率。

Curious how a specific number behaves in FP8 or BF16? Try the free converter here: https://www.bestgpusforai.com/calculators/number-to-GPU-float-converter And if you want more insights, real-world tips, and a place to discuss AI programming hardware, come hang out with us at https://www.reddit.com/r/AIProgrammingHardware. We’d love to have you. 想了解特定数值在 FP8 或 BF16 下的表现吗？请尝试此免费转换器：https://www.bestgpusforai.com/calculators/number-to-GPU-float-converter。如果您想获取更多见解、实战技巧，并寻找一个讨论 AI 编程硬件的社区，欢迎加入我们的 Reddit 频道：https://www.reddit.com/r/AIProgrammingHardware。期待您的参与。