Qwen3.5-9B-NVFP4-RTN

weili-0234/Qwen3.5-9B-NVFP4-RTN

Qwen3.5-9B-NVFP4-RTN is a text generation model from weili-0234. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.

RTN(+act-calib) NVFP4 quantization of Qwen/Qwen3.5-9B in vLLM compressed-tensors nvfp4-pack-quantized format, full W4A4 schema (static input-activation scales included; default load serves W4A4 on SM100+). The weights…

Downloads · 30 days

25% of all-time downloads

All-time downloads

Public

Parameters

6.4B

8.9 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors8.9 GB · 100%

Parameter types

How the weights are stored.

U83.5B · 54%

Try a prompt

Base models

Qwen/Qwen3.5-9B

Model card

From the Hugging Face model README

Qwen3.5-9B-NVFP4-RTN

RTN(+act-calib) NVFP4 quantization of Qwen/Qwen3.5-9B in vLLM compressed-tensors nvfp4-pack-quantized format, full W4A4 schema (static input-activation scales included; default load serves W4A4 on SM100+). The weights are byte-exact RTN — no training step was taken. 9B companion of Qwen3.5-27B-NVFP4-RTN, produced by the identical recipe.

How this checkpoint was produced — exact reproduction


Producing repo	tonyzhang-together/QATFactory @ `096f2ae`
Base model	Qwen/Qwen3.5-9B (dense `qwen3_5`, bf16)
Weights	RTN, no training: base bf16 weights rounded to the NVFP4 lattice by the same exporter used for all QAD checkpoints in this series (verified byte-identical to llm-compressor `model_free_ptq` at 9B)
Activation scales	Forward-only single-GPU calibration (`nvfp4_rtn_act_calib.py`): the training stack's own input-scale lifecycle (apply_fake_quant -> running amax -> commit at 20 step boundaries -> finalize) over 640 sequences (20x32) of the md5-pinned train corpus `openperfectblend_100k_Qwen3.5-9B_think_train.jsonl` (md5 `0406bb3a7a482352360716a1bc5e9e04`), max_seq 8192
Export	`scripts/export_nvfp4_vllm.py --source <calib ckpt> --model-assets Qwen3.5-9B` (a4schema)
Hardware	1xB200 (jbom-03), 2026-07-27
Role	Weak-PTQ baseline row of the Qwen3.5-9B standardized campaign; results filed in qatfactory-experiments `qwen3.5-9b/`

Inference

vllm >= 0.25.1 serves this checkpoint W4A4 by default on SM100+ (B200):

vllm serve weili-0234/Qwen3.5-9B-NVFP4-RTN --max-model-len 33024

Task: Text Generation
Type: qwen3_5
License: apache-2.0
Created: Jul 27, 2026
Updated: Jul 27, 2026