Qwick-3.5-9B-NVFP4

Qwick-3.5-9B-NVFP4

日本語

Qwick-3.5-9B-NVFP4 is the NVFP4 ModelOpt export of horiuchinobuyuki/Qwick-3.5-9B. Its weight files total 8.88 GB, 52.8% less than the BF16 checkpoint. It was tested with vLLM 0.23.0 and the Cutlass linear backend on an NVIDIA RTX PRO 6000 Blackwell GPU.

Developed by Nobuyuki Horiuchi.

Qwick-3.5-9B fine-tunes Qwen3.5-9B for shorter reasoning at comparable quality; this repository is its NVFP4 export. See the base model card for that motivation and the full benchmark table.

Experimental: this export is substantially smaller, but every measured screen regresses: MMLU-Pro, GPQA, IFEval, and MMMU. Review the per-benchmark results below; use the FP8 release when the additional memory saving is not required.

Evaluation scope: quantization calibration used text only. This exported checkpoint was also measured on all 900 MMMU validation rows below. The measurement is report-only, not the 10,500-row test split, and broad multimodal serving remains outside release qualification; therefore the Hub pipeline stays text-generation.

Quantization checks

These runs measure the exported weights. Completion length includes thinking and the final answer.

MMLU-Pro uses 48 questions from each of 14 subjects rather than the complete 12,032-question suite. GPQA and IFEval are complete. All three use thinking, temperature 1.0, top-p 0.95, top-k 20, min-p 0, presence penalty 1.5, repetition penalty 1.0, and at most 32,768 generated tokens. IFEval uses dataset revision 966cd89545d6b6acfd7638bc708b98261ca58e84; final content is graded while the full trace is retained. This rerun replaces every earlier temperature-zero IFEval public measurement and corrects reporting only; it did not trigger model reselection. The source BF16 checkpoint's synthetic holdout used a separately frozen temperature-zero internal release gate, which passed; it is not a public performance benchmark and was not rerun for quantization.

The BF16 model's full benchmark table and training history are available on its model card. Detailed quantization results are in EVALUATION.md and quantization_results.json.

Complete vision validation

All four rows use the complete 900-row validation split, the same official-compatible final-content grader, thinking at temperature 1.0, a 32,768-token output cap, and 131,072-token server context. Qwick BF16, FP8, and NVFP4 use the same model-independent prompt-hash seed for every item. Qwick-3.5-9B-NVFP4 minus Qwick BF16 is -4.000 pp; its subject-stratified paired descriptive interval is [-6.778, -1.222] pp and exact McNemar is p=0.0068. Under that interval, a vision regression signal was detected. Official random fallbacks, strict scores, length stops, and full-trace integrity are retained in EVALUATION.md and quantization_results.json. This report-only measurement is not the 10,500-row test split.

Public NVFP4 comparison context

The following absolute results put Qwen3.5-9B BF16, AxionML NVFP4, ig1 NVFP4, and Qwick NVFP4 under the same prompts, decode, and full-trace policy. AxionML and ig1 quantize the upstream Qwen3.5-9B; Qwick quantizes the separately trained Qwick checkpoint. The implementations, calibration sets, and unquantized modules differ, so this is not a causal comparison of quantization recipes or QAD.

MMLU-Pro is the 672-question screen and GPQA is all 198 questions. IFEval is all 541 prompts at the pinned revision with a 32,768-token cap, official-recommended general-thinking sampling, and strict final-content grading. Qwen3.5-9B BF16 reached 89.279%, with 8/541 length stops and 8 empty final answers. All earlier temperature-zero IFEval public measurements are invalid and excluded. Because the upstream card does not fully identify its harness and grader conditions, treat this as a local matched comparison rather than a direct reproduction claim.

This four-model comparison is a separate generation run from the same-seed BF16-to-NVFP4 impact table in the Quantization checks section above: its four models share one prompt-hash seed salt, whereas the impact table reuses Qwick BF16's per-prompt seeds. The two runs therefore report slightly different Qwick NVFP4 MMLU-Pro and GPQA scores; the IFEval figure is the same corrected run in both tables.

Serving measurement

The following pair used one model per GPU, 256 measured requests after 16 warmups, fixed 512-token inputs and 512-token outputs, ignore_eos, and concurrency 128.

This is a hardware- and configuration-specific saturation measurement, not a single-request latency result.

Run with vLLM

python -m pip install -r requirements.txt
vllm serve horiuchinobuyuki/Qwick-3.5-9B-NVFP4 \
  --dtype auto \
  --linear-backend cutlass \
  --language-model-only \
  --max-model-len 65536

The tested server used eager execution and unquantized KV cache. The model occupied 7.55 GiB before KV-cache allocation. Context capacity depends on the available memory and serving settings.

Quantization

• Producer: NVIDIA ModelOpt 0.45.0
• Source: the sealed BF16 Qwick checkpoint
• Quantized: 200 text-backbone linear matrices, weights and input activations
• Kept in BF16: embeddings, LM head, visual tower, and the excluded linear-attention projections
• KV cache: unquantized
• Calibration: 512 × 512 tokens from WikiText-103-v1 train, revision b08601e04326c79dfdd32d625aee71d232d685c3

Only the pinned WikiText training split was supplied for calibration; benchmark and reserved-holdout records were not calibration inputs. See provenance.json for hashes and exact export settings.

Intended use and limitations

This release is intended for text inference and deployment experiments on NVIDIA GPUs supported by the serving stack. Validate it on the prompts and hardware used in production.

• The quantized evaluation is smaller than the BF16 release evaluation: 672 MMLU-Pro questions, 198 GPQA questions, and 541 IFEval prompts.
• HMMT, JMMLU, LiveCodeBench, and the reserved synthetic holdout were not rerun after quantization. MMMU validation was rerun on the exported checkpoint with one matched sampling seed per item.
• Accuracy and generation length can vary with the backend, kernel, sampling implementation, and decode settings.
• The reported serving benchmark uses fixed synthetic token lengths on one Blackwell system; it is not a general latency or energy claim.

License and citation

Qwick-3.5-9B-NVFP4 is released under Apache-2.0. It is derived from Qwick-3.5-9B and Qwen/Qwen3.5-9B@c202236235762e1c871ad0ccb60c8ee5ba337b9a. See NOTICE for attribution.

@misc{horiuchi2026qwick35nvfp4,
  author       = {Nobuyuki Horiuchi},
  title        = {Qwick-3.5-9B-NVFP4},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/horiuchinobuyuki/Qwick-3.5-9B-NVFP4}}
}