Downloads · 30 days
38
54% of all-time downloads
lite-infer/flux.2-klein-9b-nunchaku-lite-int4-r32
flux.2-klein-9b-nunchaku-lite-int4-r32 is a text-to-image model from lite-infer. Use it when you need an image from a text prompt. It is set up for diffusers. The card lists the license as other.
Diffusers-loadable INT4 conversion of black-forest-labs/FLUX.2-klein-9B, quantized with diffuse-compressor. It loads with a plain frompretrained call — no runtime graph patches and no extra runtime package.
Downloads · 30 days
38
54% of all-time downloads
All-time downloads
70
Public
Parameters
4.7B
11.7 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors11.7 GB · 100%
How the weights are stored.
I84.4B · 92%
From the Hugging Face model README
Diffusers-loadable INT4 conversion of black-forest-labs/FLUX.2-klein-9B, quantized with
diffuse-compressor. It
loads with a plain from_pretrained call — no runtime graph patches and no extra
runtime package.
The transformer uses quant_method: nunchaku_lite, INT4 SVDQ with group size
64, rank 32, 144 SVDQ targets and 3 AWQ W4A16 targets. Six outer linears (embedders, norm_out.linear, proj_out) stay
in bf16, and the text_encoder component is BitsAndBytes 4-bit NF4 with bf16
compute. QKV projections are not fused, so this trades some speed for loading
through the stock Diffusers graph. Calibrated on 128 prompts at
4 steps, 1024x1024.
| Checkpoint | Latency | Max VRAM |
|---|---|---|
| This repo — Nunchaku Lite INT4 r32 + BNB4 text encoder | 9.96 s (stdev 0.12 s) | 8.09 GiB |
| Nunchaku Lite NVFP4 r32 + BNB4 text encoder | 2.48 s (stdev 0.00 s) | 8.32 GiB |
| FLUX.2 Klein 9B dense bf16 | 4.41 s (stdev 0.02 s) | 20.16 GiB |
NVIDIA RTX PRO 4000 Blackwell, settings as in the Run snippet below, one warmup and three measured runs, everything resident on the GPU with no offload. Latency covers denoise and decode, with text embeddings precomputed; VRAM is peak device usage.
That is 2.49x less VRAM than dense bf16, at 2.26x the latency. The latency gap is specific to this GPU, which has native FP4 tensor cores that only the NVFP4 build can use. INT4 is the build for Turing through Ada, where NVFP4 does not run at all; on Blackwell prefer the NVFP4 build.

Dense reference (left) and this build (right). Same prompt, seed, scheduler, resolution, and step count. Pixel MAE 30.14 / RMSE 45.32, versus 31.21 / 45.71 for the NVFP4 build.
Requires the Hugging Face kernels package and DIFFUSERS_TRUST_REMOTE_KERNELS=true,
plus a Diffusers build with the nunchaku_lite quantizer. INT4 needs
a Turing or newer NVIDIA GPU; Hopper is unsupported.
import torch
from diffusers import Flux2KleinPipeline
pipe = Flux2KleinPipeline.from_pretrained(
"lite-infer/flux.2-klein-9b-nunchaku-lite-int4-r32",
torch_dtype=torch.bfloat16,
).to("cuda")
image = pipe(
prompt="A glass robot tending orchids in a sunlit greenhouse, cinematic lighting, highly detailed",
generator=torch.Generator("cuda").manual_seed(12345),
width=1024,
height=1024,
num_inference_steps=4,
guidance_scale=1.0,
).images[0]
image.save("output.png")