Qwen3.5-9B-NVFP4-GPTQ

weili-0234/Qwen3.5-9B-NVFP4-GPTQ

Qwen3.5-9B-NVFP4-GPTQ is a text generation model from weili-0234. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.

GPTQ (Hessian-corrected PTQ) NVFP4 quantization of Qwen/Qwen3.5-9B in vLLM compressed-tensors nvfp4-pack-quantized format, with calibrated input global scales (W4A4-servable, default load, SM100+). Strong-PTQ baseline…

Downloads · 30 days

14% of all-time downloads

All-time downloads

618

Public

Parameters

6.4B

8.9 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors8.9 GB · 100%

Parameter types

How the weights are stored.

U83.5B · 54%

Try a prompt

Base models

Qwen/Qwen3.5-9B

Model card

From the Hugging Face model README

Qwen3.5-9B-NVFP4-GPTQ

GPTQ (Hessian-corrected PTQ) NVFP4 quantization of Qwen/Qwen3.5-9B in vLLM compressed-tensors nvfp4-pack-quantized format, with calibrated input global scales (W4A4-servable, default load, SM100+). Strong-PTQ baseline of the Qwen3.5-9B standardized campaign; 9B companion of Qwen3.5-27B-NVFP4-GPTQ, produced by the identical recipe.

How this checkpoint was produced — exact reproduction


Tooling	llm-compressor 0.12.0 (`GPTQModifier`), transformers 5.9.0
Base model	Qwen/Qwen3.5-9B (dense `qwen3_5`, bf16)
Recipe	`GPTQModifier(targets="Linear", scheme="NVFP4", ignore=["lm_head", "re:.visual.", "re:.mtp.", "re:.embed_tokens."])`; NVFP4 scheme also calibrates static per-tensor input global scales from the same pass
Calibration	Fixed 512-conversation set `calib_512.jsonl` (md5 `d1ff8cce1d785f8e51b71eb237fb7a71`): `random.Random(42).sample(range(n_rows), 512)`, sorted, from the study's OpenPerfectBlend-derived ChatML train corpus (md5 `0406bb3a7a482352360716a1bc5e9e04`); stock chat template, `max_seq_length=8192`, `num_calibration_samples=512`
Script	`gptq_9b.py` = the 27B script (`experiments/w4a4-qwen35-27b/jbom/gptq_27b.py` in the study workspace) with MODEL/OUT set to 9B
Hardware	1xB200 (jbom-03), 2026-07-27
Role	Strong-PTQ baseline row; results filed in qatfactory-experiments `qwen3.5-9b/`

Inference

vllm serve weili-0234/Qwen3.5-9B-NVFP4-GPTQ --max-model-len 33024

Task: Text Generation
Type: qwen3_5
License: apache-2.0
Created: Jul 27, 2026
Updated: Jul 27, 2026