MIT Lab's SVDQuant Achieves 4× Smaller, 3× Faster AI Models on NVIDIA Blackwell GPUs; Matches 16-bit Quality at 4-bit Precision

SVDQuant Meets NVFP4: 4× Smaller and 3× Faster FLUX with 16-bit Quality on NVIDIA Blackwell GPUs

Muyang Li*, Yujun Lin*, Zhekai Zhang*, Tianle Cai, Xiuyu Li, Junxian Guo, Enze Xie, Chenlin Meng, Jun-Yan Zhu, Song HanFebruary 21, 2025SVDQuant supports NVFP4 on NVIDIA Blackwell GPUs with 3× speedup over BF16 and better image quality than INT4. Try our interactive demo below or at https://svdquant.mit.edu/! Our code is all available at https://github.com/mit-han-lab/nunchaku. With Moore's law slowing down, hardware vendors are shifting toward low-precision inference. NVIDIA's latest Blackwell ...