flux2-klein-base-9b-nvfp4-convrot

FLUX.2-klein-base-9B — NVFP4 + ConvRot INT8 quantized

Quantized from black-forest-labs/FLUX.2-klein-base-9B, the undistilled, full-capacity 9B-parameter FLUX.2 Klein base diffusion transformer. Quantized from the repo's native/ComfyUI-format checkpoint (flux-2-klein-base-9b.safetensors, double_blocks.* key naming).

Files

Hardware requirements

Requires a Blackwell GPU (SM ≥ 10.0/12.0) for NVFP4 inference support. Tested on an RTX 5070 Ti (SM 12.0).

Quantization method

Quantized with convert_to_quant (ctq).

ctq -i flux-2-klein-base-9b.safetensors -o flux2-klein-base-9b_nvfp4_convrot_int8.safetensors \
    --nvfp4 \
    --custom-layers 'double_blocks\.(0|7)\..*\.weight$|single_blocks\.(0|1|22|23)\.linear[12]\.weight$' \
    --custom-type int8 --custom-scaling-mode row --custom-convrot --custom-convrot-group-size 256 \
    --flux2 --comfy_quant --save-quant-metadata

double_blocks and single_blocks in this architecture have unequal role-separation: double_blocks keep attention (*_attn.qkv/*_attn.proj) and MLP (*_mlp.0/*_mlp.2) as separate tensors, but single_blocks.*.linear1/linear2 fuse attention and MLP into single tensors (standard FLUX single-stream design), so a clean attention-vs-MLP split isn't possible there. This build instead routes by block position: the first/last block of each stack (8 double_blocks, 24 single_blocks) stays INT8 ConvRot; the middle blocks go NVFP4. 88 of 112 quantization-eligible tensors went to NVFP4, 24 to INT8 ConvRot (verified against the output file's own _quantization_metadata, not just the command's summary output). The --flux2 exclusion preset kept 9 layers at source BF16 precision (modulation, time embedding, the final output layer, and the image/text input embedders). Uses learned-rounding optimization (SVD/AdaRound) — not a --simple/RTN build.

Built with --custom-layers/--custom-type rather than --layer-config, sidestepping two bugs in the convert_to_quant checkout used for this build: a --layer-config entry with "format": "nvfp4" silently mapping to plain FP8, and bare --nvfp4 --layer-config (with no --custom-type/--fallback) bypassing layer-config routing and exclusions almost entirely. See silveroxides/convert_to_quant#56 for the first issue.

Verification

Confirmed to load and generate successfully end-to-end in ComfyUI (Qwen3-8B text encoder, FLUX.2 VAE), with the intended per-layer quantization format actually dispatched per the output file's own _quantization_metadata (88 NVFP4 / 24 INT8 ConvRot / 9 BF16-excluded, not just a successful command exit). Output quality has not been separately assessed against the BF16 source — only functional correctness (loads, runs, produces a coherent image matching the prompt).

License

This model is a derivative of black-forest-labs/FLUX.2-klein-base-9B, released under the FLUX Non-Commercial License v2.1 — not Apache 2.0. The full license text is included as LICENSE.md in this repo. Key terms:

• Non-commercial use only. Use, Distribution, or creation of Derivatives is permitted solely for Non-Commercial Purposes as defined in the license (personal/research/testing use with no direct or indirect payment arising from it). Commercial or production use requires a separate license from Black Forest Labs — see https://bfl.ai/licensing.

• Attribution Notice required. Anyone redistributing this model or a further derivative of it must prominently include the following notice:

  This FLUX Model is licensed by Black Forest Labs Inc. under the FLUX Non-Commercial License. Copyright Black Forest Labs Inc. IN NO EVENT SHALL BLACK FOREST LABS INC. BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH USE OF THIS MODEL.

• Modification disclosure. This file has been modified from the original via the quantization method described above — this must also be stated by anyone redistributing it further.

• The source repo does not include a separate NOTICE file, so none is carried forward here beyond the Attribution Notice above.

This quantized derivative is distributed under the same FLUX Non-Commercial License, per the terms of the source license — it is not Apache 2.0, and it may not be used for commercial or production purposes.