Qwen3.5-27B-NVFP4-GPTQ

Qwen3.5-27B-NVFP4-GPTQ

GPTQ (Hessian-corrected PTQ) NVFP4 quantization of Qwen/Qwen3.5-27B in vLLM compressed-tensors nvfp4-pack-quantized format, with calibrated input global scales (W4A4-servable, default load, SM100+). This is the strong PTQ baseline of the stage-3 W4A4-training study — same lattice and serving path as weili-0234/Qwen3.5-27B-NVFP4-RTN, but with GPTQ weight error correction and activation-scale calibration on a fixed calibration set.

How this checkpoint was produced — exact reproduction

# gptq_27b.py NVFP4  (llmcompressor==0.12.0, transformers==5.9.0)
from datasets import load_dataset
from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration
from llmcompressor import oneshot
from llmcompressor.modifiers.gptq import GPTQModifier

model = Qwen3_5ForConditionalGeneration.from_pretrained("Qwen/Qwen3.5-27B", dtype="bfloat16")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-27B")
ds = load_dataset("json", data_files="calib_512.jsonl", split="train")
recipe = GPTQModifier(
    targets="Linear",
    scheme="NVFP4",
    ignore=["lm_head", "re:.*visual.*", "re:.*mtp.*", "re:.*embed_tokens.*"],
)
oneshot(model=model, tokenizer=tokenizer, dataset=ds, recipe=recipe,
        max_seq_length=8192, num_calibration_samples=512)
model.save_pretrained("Qwen3.5-27B-NVFP4-GPTQ", save_compressed=True)
tokenizer.save_pretrained("Qwen3.5-27B-NVFP4-GPTQ")

Serving

vllm serve weili-0234/Qwen3.5-27B-NVFP4-GPTQ    # W4A4 (native NVFP4 activations, SM100+)

<!-- eval-results-kld -->

Serving-KL vs BF16 (2026-07-24)

Mean top-20 token KL vs the BF16 teacher (Qwen/Qwen3.5-27B) over a held-out 256-conversation corpus (818,944 scored tokens), vLLM on 1×B200, W4A4 serving.

Caveat — this artifact does NOT beat RTN on serving-KL for NVFP4. Unlike MXFP4 (where GPTQ recovers ~23% of the RTN gap), the llm-compressor NVFP4 pipeline lands 2.6× worse than the QATFactory RTN export at W4A4 serving. The leading hypothesis is the input-activation global-scale calibration (static observer scales here vs running-max scales in the RTN export) dominating NVFP4's small weight-quantization error; the GPTQ weight update itself may be neutral-to-positive. Treat this checkpoint as a faithful record of the standard llm-compressor GPTQ-NVFP4 recipe at 27B rather than as a strengthened PTQ baseline.

<!-- eval-results-kld -->

A 7-benchmark evaluation (RULER, AIME'25 avg@4, GPQA-Diamond, MMLU-Pro, HumanEval, IFEval, Pile-10k/WikiText perplexity) against the RTN and QAD checkpoints is in progress and will be added to this card.