Krea2-Turbo-mixed-NVFP4

Krea2-Turbo-mixed-NVFP4

Mixed 8-bit microscaling quantization of Krea-2-Turbo, generated with convert_to_quant.

• Format: Mixed NFVP4 and MXFP8 with some layers preserved as BF16.
• Size: 8.8G GB (−66% vs BF16).
• Inference: ComfyUI + comfy-kitchen, Blackwell GPU (RTX 50xx / B100 / B200).

<table> <tr> <td align="center"> <a href="Krea2-Turbo-mixed-NVFP4/blob/main/assets/nvfp4.png"><img src="assets/nvfp4.png" style="width: 66%; height: auto;"></a> </td> <td align="center"> <a href="Krea2-Turbo-mixed-NVFP4/blob/main/assets/bf16.png"><img src="assets/bf16.png" style="width: 66%; height: auto;"></a> </td> </tr> <tr> <td align="center"><strong>NVFP4</strong></td> <td align="center"><strong>BF16</strong></td> </tr> </table>

Quantization

The model weights have been partially quantized to NVFP4 (NVIDIA Floating Point 4-bit) and MXFP8, quantization formats supported on NVIDIA Blackwell architecture GPUs.

The following convert_to_quant parameters where used. This conversion takes about 1 hour on an RTX 5060 TI.

$ convert_to_quant -i krea2_turbo_bf16.safetensors \
    --nvfp4 \
    --krea2 \
    --comfy_quant \
    --save-quant-metadata \
    --custom-type mxfp8 \
    --custom-layers \
      "blocks\.(0|1|2|24|25|26)\.attn\.(wq|wk|wv|wo)\.weight|blocks\.(0|1|2|25|26|27)\.attn\.gate\.weight|blocks\.(0|1|2|3|25|26|27)\.mlp\.gate\.weight|txtfusion\.layerwise_blocks\.(0|1)\.attn\.(wq|wk|wv|wo|gate)\.weight|txtfusion\.layerwise_blocks\.(0|1)\.mlp\.gate\.weight|txtfusion\.refiner_blocks\.(0|1)\.attn\.(wq|wk|wv|gate)\.weight|txtfusion\.refiner_blocks\.(0|1)\.mlp\.(gate|up)\.weight|txtfusion\.refiner_blocks\.0\.mlp\.down\.weight" \
    --exclude-layers \
      "blocks\.27\.attn\.(wq|wk|wv)\.weight|txtfusion\.refiner_blocks\.(0|1)\.attn\.wo\.weight|txtfusion\.refiner_blocks\.1\.mlp\.down\.weight" \
    --num-iter 4000 \
    --top-p 0.35 \
    --calib-samples 8192 \
    --scale-optimization iterative \
    --scale-refinement 2 \
    --extract-lora \
    --lora-rank 64 \
    --lora-target "attn\.(wo|gate)\.weight|mlp\.(gate|down)\.weight" \
    -o krea2_turbo_mixed_nvfp4.safetensors