Mistral-Nemo-Instruct-2407-NVFP4-4over6

Quantized NVFP4 version of Mistral-Nemo-Instruct-2407 with Four Over Six adaptive block scaling, created to compare against my hybrid quant. Made with the same version of llm-compressor and compressed-tensors, using the same calibration data, to isolate the variables as much as possible.

Resources

• Paper: Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling
• Repository: GitHub - mit-han-lab/fouroversix

Method

Four Over Six (4/6) is a modification to the block-scaled NVFP4 quantization algorithm that yields reduced quantization error. Unlike integer formats, floating point formats have non-uniform step sizes which create larger quantization error on larger values. 4/6 takes advantage of this by adaptively scaling some blocks to smaller FP4 values, making the distribution of representable values more uniform and reducing quantization error for near-maximal values.

Citation

@misc{cook2025sixaccuratenvfp4quantization,
      title={Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling},
      author={Jack Cook and Junxian Guo and Guangxuan Xiao and Yujun Lin and Song Han},
      year={2025},
      eprint={2512.02010},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2512.02010},
}