FLUX.1-dev-4bit-gguf

FLUX.1-dev 4-bit GGUF (Q4_K_S)

Mirror of City96/FLUX.1-dev-gguf for fast personal access. Please credit the original creators below if you use this repo.

License (IMPORTANT)

This is a quantized version of FLUX.1 [dev] by Black Forest Labs, released under the FLUX.1 [dev] Non-Commercial License.

• Non-commercial use only
• Research and personal use OK
• No commercial deployment
• No redistribution as a paid service

See the original license link above for full terms.

Credits

• Original model: Black Forest Labs (black-forest-labs/FLUX.1-dev)
• GGUF quantization: City96 (City96/FLUX.1-dev-gguf)
• This mirror: uploaded for personal and community convenience

Specs

• Quantization: Q4_K_S (4-bit)
• File size: about 6.9 GB
• Inference VRAM: about 6-7 GB with CPU offload
• Works on: NVIDIA T4, RTX 3060+, A100, H100

Usage (Diffusers)

import torch
from diffusers import FluxPipeline, FluxTransformer2DModel

transformer = FluxTransformer2DModel.from_pretrained(
    "dgflash/FLUX.1-dev-4bit-gguf",
    gguf_file="flux1-dev-Q4_K_S.gguf",
    torch_dtype=torch.float16,
)

pipe = FluxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev",
    transformer=transformer,
    torch_dtype=torch.float16,
    token="YOUR_HF_TOKEN",
)
pipe.enable_model_cpu_offload()
pipe.vae.enable_tiling()

image = pipe(
    prompt="A golden pagoda at sunset, cinematic photography, 8k",
    num_inference_steps=20,
    guidance_scale=3.5,
    width=1024, height=1024,
).images[0]
image.save("output.png")

Note: FLUX.1-dev is gated. Accept the license on Hugging Face once (free) before using the pipeline components.

Tips for T4 (16 GB) users

• Always use enable_model_cpu_offload()
• Start with 20 steps and guidance 3.5
• First image takes 2-4 minutes, later images about 1-2 minutes