Qwen3-4B-W2.79-QAD

Qwen3-4B — W2.79, QAD checkpoint

The quantization-aware-distillation checkpoint that on-policy distillation starts from, for the W2.79 arm. This is a starting point for training, not a finished model.

Code and recipe: MingZwhy/QAOPD.

This is a latent checkpoint — read before loading

The tensors here are bf16 and not quantized. Quantization-aware training keeps high-precision master weights and applies the quantizer inside the forward pass; what gets saved is the master copy. Loading this file directly gives you an unquantized model. It will not error, and it will score higher than the W2.79 model does.

Use it as STUDENT_MODEL for the OPD stage, which supplies the quantizer configuration itself:

BITWIDTH=w2.79 STUDENT_MODEL=<this checkpoint> bash scripts/opd/run_math.sh

For a model you can load and evaluate, use the recovered checkpoint MingZwhy/Qwen3-4B-W2.79-QAOPD instead.

Quantization

License

Apache-2.0, inherited from Qwen3-4B.