WarpQuant-Qwen3.5-4B-R16E4V4

WarpQuant Qwen3.5-4B R16E4V4

Qwen3.5-4B quantized with signed Hadamard rotation, block-GPTQ, and Output-Fisher weak-column recovery. The projection layers use a 3.5-bpw INT3 base, selected columns are restored in BF16, and the token embedding and vision weights use group-128 INT4.

Technical report · GitHub · Text-only model

Format

The repository stores the quantized values in BF16-compatible safetensors for direct Transformers loading. The bpw value is the analytical packed payload including codes, scales, recovery values, and column indices.

Evaluation

Usage

Qwen3.5 currently requires the latest Transformers main branch:

pip install "transformers @ git+https://github.com/huggingface/transformers.git@main"

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "HarimxChoi/WarpQuant-Qwen3.5-4B-R16E4V4"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

Citation

@misc{choi2026warpquant,
  author = {Harim Choi},
  title = {WarpQuant: Dual-Domain LLM Quantization via Hadamard Rotation and Output-Fisher Sensitivity},
  year = {2026},
  url = {https://harimxchoi.github.io/projects/warpquant/}
}