NVIDIA / Model-Optimizer
NVIDIA Model Optimizer
NVIDIA Model Optimizer (referred to as Model Optimizer, or ModelOpt) is a library comprising state-of-the-art model optimization techniques including quantization, pruning, Neural Architecture Search (NAS), distillation, speculative decoding and sparsity to accelerate models.
NVIDIA Model Optimizer（简称 Model Optimizer 或 ModelOpt）是一个包含尖端模型优化技术的库，其中包括量化、剪枝、神经架构搜索 (NAS)、蒸馏、投机解码和稀疏化，旨在加速模型运行。
[Input] Model Optimizer currently supports inputs of a Hugging Face, PyTorch or ONNX model.
[输入] Model Optimizer 目前支持 Hugging Face、PyTorch 或 ONNX 模型作为输入。
[Optimize] Model Optimizer provides Python APIs for users to easily compose the above model optimization techniques and export an optimized quantized checkpoint. Model Optimizer is also integrated with NVIDIA Megatron-Bridge, Megatron-LM and Hugging Face Accelerate for training required inference optimization techniques.
[优化] Model Optimizer 提供 Python API，使用户能够轻松组合上述模型优化技术，并导出优化后的量化检查点 (checkpoint)。Model Optimizer 还与 NVIDIA Megatron-Bridge、Megatron-LM 和 Hugging Face Accelerate 集成，用于训练所需的推理优化技术。
[Export for deployment] Seamlessly integrated within the NVIDIA AI software ecosystem, the quantized checkpoint generated from Model Optimizer is ready for deployment in downstream inference frameworks like SGLang, TensorRT-LLM, TensorRT, or vLLM. The unified Hugging Face export API now supports both transformers and diffusers models.
[导出部署] 作为 NVIDIA AI 软件生态系统的一部分，由 Model Optimizer 生成的量化检查点可无缝集成，并准备好部署到 SGLang、TensorRT-LLM、TensorRT 或 vLLM 等下游推理框架中。统一的 Hugging Face 导出 API 现已同时支持 transformers 和 diffusers 模型。
Latest News / 最新动态
[2026/09/16] End-to-end W4A4 NVFP4 + QAD tutorial for Qwen3.6-35B-A3B: NVFP4 W4A4 PTQ plus quantization-aware distillation, reaching up to 1.30x vLLM throughput over BF16 and 3.1x smaller checkpoints while recovering the accuracy W4A4 costs.
[2026/09/16] Qwen3.6-35B-A3B 的端到端 W4A4 NVFP4 + QAD 教程：通过 NVFP4 W4A4 PTQ 加上量化感知蒸馏，在恢复 W4A4 精度损失的同时，实现了比 BF16 高出 1.30 倍的 vLLM 吞吐量，并将检查点大小缩小了 3.1 倍。
[2026/09/09] BLOG: Improving NVFP4 Accuracy with Local-Hessian Weight Scales
[2026/09/09] 博客：利用局部海森权重缩放 (Local-Hessian Weight Scales) 提升 NVFP4 精度
[2026/08/24] BLOG: AutoQuantize: A Fast Automatic Mixed-Precision Assignment
[2026/08/24] 博客：AutoQuantize：一种快速的自动混合精度分配方法
[2026/08/17] BLOG: Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer: Learn how quantization-aware distillation recovers accuracy from aggressive NVFP4 quantization while reducing model size and increasing throughput.
[2026/08/17] 博客：使用 NVIDIA Model Optimizer 开发 Nemotron 3.5 Lightning NVFP4：了解量化感知蒸馏如何从激进的 NVFP4 量化中恢复精度，同时减小模型尺寸并提高吞吐量。
[2026/06/26] BLOG: Creating the NVIDIA Nemotron 3 Ultra NVFP4 Checkpoint with NVIDIA Model Optimizer: How we quantized Nemotron 3 Ultra (550B) to NVFP4 with Model Optimizer — up to 5.9× higher decode-heavy inference throughput than GLM-5.1 754B FP4 while matching BF16 accuracy. NVFP4 Checkpoint on Hugging Face.
[2026/06/26] 博客：使用 NVIDIA Model Optimizer 创建 NVIDIA Nemotron 3 Ultra NVFP4 检查点：我们如何使用 Model Optimizer 将 Nemotron 3 Ultra (550B) 量化为 NVFP4——在匹配 BF16 精度的同时，解码密集型推理吞吐量比 GLM-5.1 754B FP4 高出 5.9 倍。Hugging Face 上的 NVFP4 检查点。
[2026/05/27] End-to-end Optimization tutorial for Nemotron-3-Nano-30B-A3B: Pruning + two-phase distillation + FP8 quantization achieving 2.6× vLLM throughput and 2.6× memory reduction.
[2026/05/27] Nemotron-3-Nano-30B-A3B 的端到端优化教程：通过剪枝 + 两阶段蒸馏 + FP8 量化，实现了 2.6 倍的 vLLM 吞吐量和 2.6 倍的内存缩减。
[2026/05/13] Puzzletron: A new algorithm for heterogeneous pruning & NAS of LLM and VLM models.
[2026/05/13] Puzzletron：一种用于 LLM 和 VLM 模型异构剪枝与 NAS 的新算法。
[2026/04/15] Customer story: Domyn compresses Colosseum-355B → 260B using ModelOpt’s Minitron pruning + distillation
[2026/04/15] 客户案例：Domyn 使用 ModelOpt 的 Minitron 剪枝 + 蒸馏技术将 Colosseum-355B 压缩至 260B。
[2026/03/17] Customer story: Bielik.AI builds Bielik Minitron 7B (33% smaller, 50% faster, 90% quality retained) using ModelOpt’s Minitron pruning + distillation
[2026/03/17] 客户案例：Bielik.AI 使用 ModelOpt 的 Minitron 剪枝 + 蒸馏技术构建了 Bielik Minitron 7B（体积缩小 33%，速度提升 50%，保留了 90% 的质量）。
[2026/03/11] Model Optimizer quantized Nemotron-3-Super checkpoints are available on Hugging Face for download: FP8, NVFP4. Learn more in the Nemotron 3 Super release blog. Check out how to quantize Nemotron 3 models for deployment acceleration here.
[2026/03/11] Model Optimizer 量化的 Nemotron-3-Super 检查点现已在 Hugging Face 上提供下载：FP8 和 NVFP4。在 Nemotron 3 Super 发布博客中了解更多信息。点击此处查看如何量化 Nemotron 3 模型以加速部署。
[2026/03/11] NeMo Megatron Bridge now supports Nemotron-3-Super quantization (PTQ and QAT) and export workflows using the Model Optimizer library. See the Quantization (PTQ and QAT) guide for FP8/NVFP4 quantization and HF export instructions.
[2026/03/11] NeMo Megatron Bridge 现已支持使用 Model Optimizer 库进行 Nemotron-3-Super 量化（PTQ 和 QAT）及导出工作流。请参阅量化（PTQ 和 QAT）指南，获取 FP8/NVFP4 量化和 HF 导出说明。
Install / 安装
To install stable release packages for Model Optimizer:
如需安装 Model Optimizer 的稳定发布包：