flux.1-dev-nunchaku-lite-nvfp4-r32

FLUX.1 Dev Nunchaku Lite NVFP4 r32

Diffusers-loadable conversion of:

• Base model: black-forest-labs/FLUX.1-dev
• Source repo: nunchaku-ai/nunchaku-flux.1-dev
• Source checkpoint: svdq-fp4_r32-flux.1-dev.safetensors

The transformer uses quant_method: nunchaku_lite, NVFP4 SVDQ with group size 16, runtime rank 64, 418 SVDQ targets, and 76 AWQ W4A16 targets. The CLIP encoder is copied from the base model and T5 text_encoder_2 is BitsAndBytes 4-bit NF4. Fused QKV modules are split in logical tensor layout; single-block proj_out is merged from attention and MLP projections; low-rank tensors are logically padded to rank 64. NVFP4 outer scales are reconciled without overflowing FP8 group scales.

Benchmark

RTX 5090, 1024×1024, 28 steps, guidance scale 3.5, one warmup and three measured runs, full GPU placement. VRAM is peak total device usage sampled with nvidia-smi, including allocations outside PyTorch's caching allocator. The native row uses the base model's BF16 T5 encoder.

Output Comparison

Both images use the same prompt, seed 0, scheduler, resolution, and step count. The native and converted NVFP4 images use identical inputs.

Run

Requires the Hugging Face kernels package and a Blackwell NVIDIA GPU for NVFP4 kernels.

import torch
from diffusers import FluxPipeline

pipe = FluxPipeline.from_pretrained(
    "lite-infer/flux.1-dev-nunchaku-lite-nvfp4-r32",
    torch_dtype=torch.bfloat16,
).to("cuda")

image = pipe(
    prompt='A cinematic photograph of a red fox standing in a misty forest at sunrise, detailed fur, volumetric light',
    generator=torch.Generator("cuda").manual_seed(0),
    width=1024,
    height=1024,
    num_inference_steps=28,
    guidance_scale=3.5,
).images[0]
image.save("output.png")