NVIDIA-Nemotron-3-Nano-30B-A3B-EXL3

NVIDIA Nemotron 3 Nano 30B A3B EXL3 quants

Two EXL3 variants of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16, converted with ExLlamaV3 1.4.2. These are community-made quantizations by ultimatechris, not NVIDIA releases.

Available quantizations

• 4.0 bpw HQ, 16.19 GiB
• 3.0 bpw HQ, 12.48 GiB

Quantization

• Converter: ExLlamaV3 1.4.2
• Recipe: high-quality (-hq) EXL3 conversion
• Conversion hardware: one NVIDIA A100 80GB

The HQ pass puts the large expert matrices at the target bitrate and leaves selected shared/mixer projections and the language-model head at higher bitrates. Each directory contains a quantization_config.json with the exact tensor layout.

This is a mixture-of-experts model. We did not compare expert IDs between BF16 and EXL3, so this release makes no claim that routing is bit-for-bit identical. A routing comparison can be a separate diagnostic experiment.

Runtime

Use an ExLlamaV3-compatible frontend such as TabbyAPI. Each variant directory includes the tokenizer, chat template, configuration, and custom-code files needed by the base model.

Point the frontend at the 4.0 bpw directory (16.19 GiB) or the 3.0 bpw directory (12.48 GiB). Runtime speed and maximum context depend on the frontend, GPU, context length, and batch settings.

Attribution and limitations

The base model was developed and released by NVIDIA. These quantized files retain its capabilities, limitations, and usage requirements. Read the base model card before deployment.

EXL3 attribution

The EXL3 format and conversion tooling are provided by TurboDerp's ExLlamaV3.