gemma-3-12b-it-int8-convrot

Gemma 3 12B IT - INT8 ConvRot

INT8 ConvRot quantization of Gemma 3 12B IT for ComfyUI text-encoder use. The original 24.38 GB BF16 checkpoint is reduced to 14.08 GB.

Conversion

• Tool: silveroxides/convert_to_quant
• Format: INT8 row-wise with embedded ConvRot metadata
• ConvRot group size: 256
• Method: learned rounding (AdaRound) with low-memory streaming conversion
• Quantized: all linear projections in text-transformer blocks 1-46 (322 matrices; ~10.31B parameters)

The token embedding table, full vision tower, and first and last text-transformer blocks remain BF16.

Command

ctq -i <input-model>.safetensors -o <output-model>.safetensors `
  --int8 --scaling_mode row `
  --convrot --convrot-group-size 256 `
  --comfy_quant --save-quant-metadata `
  --low-memory --device cuda `
  --exclude-layers '(^model\.embed_tokens\.weight$|^vision_model\.|^model\.layers\.(0|47)\.)' `
  --verbose NORMAL

Quantization is lossy, so outputs are not bit-identical to the original BF16 checkpoint.