kroma-v0.3-base-INT8-convrot

Stabhappy/kroma-v0.3-base-INT8-convrot

INT8 ConvRot quantized weights for Kroma v0.3 base (lodestones/Kroma, Krea 2 / K2 DiT architecture), for fast, low-VRAM inference in ComfyUI via the native INT8 loader.

Quantized derivative of kroma-v0.3-base.safetensors. Not an official Krea/Kroma release. Original weights © their respective owners under the Krea 2 Community License.

File

Original kroma-v0.3-base.safetensors is 51.3 GB (51,280,336,920 B), F32, 430 tensors, Krea 2 DiT (hidden 6144). ~3.6× reduction.

Quantization method

• Format: INT8 ConvRot — row-wise INT8 weights with group-wise Hadamard rotation (group size 256) applied to smooth activation outliers before quantization. Targets near-lossless quality (≈ GGUF-Q8 class).
• Optimization: SVD learned rounding + bias correction (calibration samples: 3072, tool default).
• Output: native ComfyUI INT8 — comfy_quant metadata embedded per-layer. Loads with the standard model loader, no custom node required.
• Inference compute: INT8 tensor cores (torch._int_mm) + Triton kernels.

Preserved-layer output dtype policy (new in this release)

Since ctq v1.3.2, unquantized preserved layers are not unconditionally stored F32. The provider default (--krea2) is:

• BF16 for unquantized 2D weights, except
• F32 for krea2-mapped exception weights: first, last.linear, tmlp.0/2, tproj.1, txtfusion.projector, txtmlp.1/3 .weight
• Quantized (INT8) layers record BF16 compute dtype.

For this checkpoint that lands as: 224 INT8-transformed block tensors, 32 BF16 txtfusion layerwise weights (~0.69 GB), and 34 F32 preserved tensors.

This deliberately differs from the v0.2 release (which stored all preserved layers F32) and reflects silveroxides' recommended current default.

Command

ctq -i kroma-v0.3-base.safetensors \
    -o kroma-v0.3-base-int8convrot.safetensors \
    --int8 --scaling_mode row --convrot \
    --comfy_quant --save-quant-metadata \
    --krea2 --low-memory

Tool: silveroxides/convert_to_quant v1.3.3 (ctq). Environment: PyTorch 2.13.0+cu130, Triton 3.7.1, Python 3.13.

Layer strategy (Krea 2 profile, --krea2)

• INT8-quantized: the 28 main DiT blocks (blocks.0–27) — attention (wq/wk/wv/wo/gate) and MLP (gate/up/down).
• Kept high-precision: first, last (incl. norm/linear/modulation), tmlp, tproj, txtfusion (layerwise + refiner blocks), txtmlp — the outlier-sensitive embedding / time / text-fusion layers (F32 exceptions or BF16 per the dtype policy above).

Changelog vs v0.2 release

• ctq 1.3.1 → 1.3.3.
• Preserved-layer dtype policy introduced (v1.3.2): non-exception preserved 2D weights now store BF16 instead of F32; krea2 exception weights stay F32.
• Everything else identical (same --krea2 profile, same layer exclusions, same INT8 ConvRot method).

Requirements

• ComfyUI with native INT8 support (Comfy-Org commit 1a510f04 or recent release).
• NVIDIA GPU with sufficient INT8 TOPS — RTX 30-series and up.
• Text encoder: qwen3vl_4b_fp8_scaled.safetensors → models/text_encoders/
• VAE: qwen_image_vae.safetensors → models/vae/
• Triton installed in the ComfyUI environment.

Usage

Load kroma-v0.3-base-int8convrot.safetensors with the standard model loader and run the normal Kroma graph (CLIPLoader → CLIPTextEncode → KSampler → VAEDecode).

Notes

• This file is the DiT transformer only; text encoder and VAE load separately as above.
• LoRA: works via the standard loader. For best LoRA quality, merge the LoRA into the base before quantizing (pre-lora) rather than stacking onto the INT8 weights.
• ConvRot = a QuaRot-derived plug-and-play rotation; small inference overhead in exchange for a quality gain over plain row-wise INT8.
• Retains the original model's license and attribution requirements; disclose AI-generated outputs where required.

Credits

lodestones/Kroma (base model) · silveroxides/convert_to_quant (INT8 ConvRot method & ctq tool) · BobJohnson24/ComfyUI-INT8-Fast (ConvRot lineage).