flux.1-kontext-dev-nunchaku-lite-int4-r32

FLUX.1 Kontext Dev Nunchaku Lite INT4 r32

Diffusers-loadable conversion of:

• Base model: black-forest-labs/FLUX.1-Kontext-dev
• Source repo: nunchaku-ai/nunchaku-flux.1-kontext-dev
• Source checkpoint: svdq-int4_r32-flux.1-kontext-dev.safetensors

The CLIP text encoder is copied from the base model, and the T5 text_encoder_2 is saved with BitsAndBytes 4-bit NF4.

Benchmark

Output Comparison

The comparison uses the same input image, prompt, seed, scheduler, and 28-step settings for this converted checkpoint and its original Nunchaku safetensors baseline. The converted checkpoint runs through the stock Diffusers graph with the Nunchaku Lite quantizer — no fused-GELU patches and no split-linear patches.

Run

Requires a Turing, Ampere, Ada, or Blackwell NVIDIA GPU (INT4 kernels are not supported on Hopper) and the Hugging Face kernels package.

import torch
from PIL import Image
from diffusers import FluxKontextPipeline

model_path = "lite-infer/flux.1-kontext-dev-nunchaku-lite-int4-r32"
image_path = "input.png"
output_path = "output.png"

pipe = FluxKontextPipeline.from_pretrained(model_path, torch_dtype=torch.bfloat16)
pipe.to("cuda")

image = Image.open(image_path).convert("RGB")
prompt = "Make Pikachu hold a sign that says 'Nunchaku is awesome', yarn art style, detailed, vibrant colors"

result = pipe(
    image=image,
    prompt=prompt,
    generator=torch.Generator(device="cuda").manual_seed(1),
    true_cfg_scale=1.0,
    guidance_scale=2.5,
    num_inference_steps=28,
    max_area=512 * 512,
)

result.images[0].save(output_path)