qwen3vl_8b_int8_convrot

Qwen3-VL 8B - INT8 ConvRot

INT8 ConvRot quantization of Qwen3-VL 8B for ComfyUI text-encoder and multimodal conditioning use.

Available versions

New: BF16-vision hybrid

The hybrid preserves the original quant's language tensors byte-for-byte and replaces the complete vision tower, including biases, embeddings and mergers, with the BF16 tower from the ComfyUI Qwen Image 2.1 preparation checkpoint. Vision quantization scales and metadata were removed; language ConvRot settings are unchanged. Every copied tensor was verified against its source.

Recommended as the quality-first option for Qwen Image 2.1 reference-image workflows: it avoids vision quantization error while retaining our BF16 language blocks 0-2 (DeepStack integration) and 35 (the final block whose unnormalized output Qwen Image 2.1 consumes). In the current ComfyUI integration, select CLIP type qwen_image.

The hybrid adds approximately 380 MB. Text-only encoding is unchanged because it does not use the vision tower. Better generated-image quality has not been established by comparative testing; this is a precision-preserving choice, not a guaranteed improvement.

Original version conversion

• Tool: silveroxides/convert_to_quant
• Format: INT8 row-wise with embedded ConvRot metadata
• Language ConvRot group size: 256
• Vision ConvRot group size: 16
• Method: learned rounding (AdaRound) with low-memory streaming conversion
• Quantized: language-transformer blocks 3-34 and vision-transformer blocks 1-25 (324 matrices; ~6.55B parameters, or 74.8% of matrix parameters)

The token embedding, LM head, language blocks 0-2 and 35, vision boundary blocks, vision embeddings, and vision merger components remain BF16.

Layer and group selection

Qwen3-VL injects three DeepStack vision representations into language blocks 0-2, so those multimodal integration blocks remain BF16. Language block 35 is retained as the final block for general Qwen3-VL use.

Vision blocks 0 and 26 form the input and output boundaries of the 27-block vision tower. Keeping these boundary layers in BF16 is a conservative quality-preservation heuristic rather than a model-author-specified ConvRot requirement.

The language matrices are compatible with ConvRot group size 256. The vision tower uses input dimensions including 1152 and 4304, which are not divisible by 256 but are divisible by 16. A custom group size of 16 ensures that the selected vision weights receive ConvRot instead of silently falling back to plain row-wise INT8.

Original version command

ctq -i <input-model>.safetensors -o <output-model>.safetensors `
  --int8 --scaling_mode row `
  --convrot --convrot-group-size 256 `
  --custom-layers '^model\.visual\.blocks\.(?:[1-9]|1[0-9]|2[0-5])\.' `
  --custom-type int8 `
  --custom-scaling-mode row `
  --custom-convrot `
  --custom-convrot-group-size 16 `
  --comfy_quant --save-quant-metadata `
  --low-memory --device cuda `
  --exclude-layers '(^model\.embed_tokens\.weight$|^lm_head\.weight$|^model\.layers\.(0|1|2|35)\.|^model\.visual\.blocks\.(0|26)\.|^model\.visual\.(pos_embed|patch_embed|merger|deepstack_merger_list)(\.|$))' `
  --verbose NORMAL

Quantization is lossy, so outputs are not bit-identical to the original BF16 checkpoint.