README

!Banner image

 NVIDIA Model Optimizer





Documentation |
Roadmap |
Announcement Blogs



______________________________________________________________________

NVIDIA Model Optimizer (referred to as Model Optimizer, or ModelOpt) is a library comprising state-of-the-art model optimization techniques including quantization, pruning, Neural Architecture Search (NAS), distillation, speculative decoding and sparsity to accelerate models.

[Input] Model Optimizer currently supports inputs of a Hugging Face, PyTorch or ONNX model.

[Optimize] Model Optimizer provides Python APIs for users to easily compose the above model optimization techniques and export an optimized quantized checkpoint.
Model Optimizer is also integrated with NVIDIA Megatron-Bridge, Megatron-LM and Hugging Face Accelerate for training required inference optimization techniques.

[Export for deployment] Seamlessly integrated within the NVIDIA AI software ecosystem, the quantized checkpoint generated from Model Optimizer is ready for deployment in downstream inference frameworks like SGLang, TensorRT-LLM, TensorRT, or vLLM. The unified Hugging Face export API now supports both transformers and diffusers models.

 Latest News

- [2026/10/28] BLOG: GLM-5.3-Flash at 2-3 Bits per Expert Weight: IQ2_S PTQ Recipes and QAD: IQ2_S experts with NVFP4 down projections match the FP8 checkpoint without training, at less than half its size; QAD recovers long-context accuracy on the all-IQ2_S model.
- [2026/10/15] NVIDIA Nemotron 3.5 Super VL 120B-A12B NVFP4 checkpoint: Quantized with Model Optimizer using post-training quantization (PTQ) and quantization-aware distillation (QAD). For details, see the Quantization section (Section 4) of the technical report.
- [2026/10/08] BLOG: AutoQuant KV: Automatic Mixed-Precision KV Cache Quantization: Per-layer KV format search, GEMM composition, and a Qwen3-8B accuracy case study.
- [2026/09/16] BLOG: Scale Learning: Training NVFP4 Block Scales to Recover Accuracy
- [2026/09/16] End-to-end W4A4 NVFP4 + QAD tutorial for Qwen3.6-35B-A3B: NVFP4 W4A4 PTQ plus quantization-aware distillation, reaching up to 1.30x vLLM throughput over BF16 and 3.1x smaller checkpoints while recovering the accuracy W4A4 costs.
- [2026/09/09] BLOG: Improving NVFP4 Accuracy with Local-Hessian Weight Scales
- [2026/08/24] BLOG: AutoQuantize: A Fast Automatic Mixed-Precision Assignment
- [2026/08/17] BLOG: Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer: Learn how quantization-aware distillation recovers accuracy from aggressive NVFP4 quantization while reducing model size and increasing throughput.
- [2026/06/26] BLOG: Creating the NVIDIA Nemotron 3 Ultra NVFP4 Checkpoint with NVIDIA Model Optimizer: How we quantized Nemotron 3 Ultra (550B) to NVFP4 with Model Optimizer — up to 5.9× higher decode-heavy inference throughput than GLM-5.1 754B FP4 while matching BF16 accuracy. NVFP4 Checkpoint on Hugging Face.
- [2026/05/27] End-to-end Optimization tutor