flux.1-kontext-dev-nunchaku-lite-nvfp4-r32

FLUX.1 Kontext Dev Nunchaku Lite NVFP4 r32

Diffusers-loadable conversion of:

• Base model: black-forest-labs/FLUX.1-Kontext-dev
• Source repo: nunchaku-ai/nunchaku-flux.1-kontext-dev
• Source checkpoint: svdq-fp4_r32-flux.1-kontext-dev.safetensors

The CLIP text encoder is copied from the base model, and the T5 text_encoder_2 is saved with BitsAndBytes 4-bit NF4.

Benchmark

The native engine's latency advantage comes from hand-written CUDA kernels and a purpose-built pipeline path, versus the Diffusers Nunchaku Lite quantizer's more general module-replacement path — text encoder choice barely affects per-run latency since T5 encoding happens once per call, not per step. Its higher VRAM here is entirely the bf16 T5 (~9.5 GB) vs. this repo's NF4 T5 (~5 GB); with a BNB4 T5 substituted into the native pipeline instead, its peak VRAM measured 14.13 GiB, matching the converted pipeline.

Output Comparison

The comparison uses the same input image, prompt, seed, scheduler, and 28-step settings for this converted checkpoint and the original Nunchaku safetensors checkpoint run through the native Nunchaku engine with its stock bf16 T5 (same GPU and session as the benchmark above).

Run

Requires a Blackwell or newer NVIDIA GPU (NVFP4 kernels) and the Hugging Face kernels package.

import torch
from PIL import Image
from diffusers import FluxKontextPipeline

model_path = "lite-infer/flux.1-kontext-dev-nunchaku-lite-nvfp4-r32"
image_path = "input.png"
output_path = "output.png"

pipe = FluxKontextPipeline.from_pretrained(model_path, torch_dtype=torch.bfloat16)
pipe.to("cuda")

image = Image.open(image_path).convert("RGB")
prompt = "Make Pikachu hold a sign that says 'Nunchaku is awesome', yarn art style, detailed, vibrant colors"

result = pipe(
    image=image,
    prompt=prompt,
    generator=torch.Generator(device="cuda").manual_seed(1),
    true_cfg_scale=1.0,
    guidance_scale=2.5,
    num_inference_steps=28,
    max_area=512 * 512,
)

result.images[0].save(output_path)