flux_qint_8bit

flux_qint_8bit

Pre-quantized FLUX models using optimum-quanto for memory-efficient inference on consumer hardware.

Summary

Available Quantizations

---

Model Details

FLUX.1 Canny [dev]

Source: black-forest-labs/FLUX.1-Canny-dev

Pipeline: FluxControlPipeline

Use case: 12B canny edge-guided generation model

FLUX.1 Depth [dev]

Source: black-forest-labs/FLUX.1-Depth-dev

Pipeline: FluxControlPipeline

Use case: 12B depth-guided generation model

FLUX.1 Fill [dev]

Source: black-forest-labs/FLUX.1-Fill-dev

Pipeline: FluxFillPipeline

Use case: 12B inpainting/outpainting model

FLUX.1 Kontext [dev]

Source: black-forest-labs/FLUX.1-Kontext-dev

Pipeline: FluxKontextPipeline

Use case: 12B image editing model (in-context generation)

FLUX.1 [dev]

Source: black-forest-labs/FLUX.1-dev

Pipeline: FluxPipeline

Use case: 12B high-quality generation model (guidance distilled)

FLUX.1 [schnell]

Source: black-forest-labs/FLUX.1-schnell

Pipeline: FluxPipeline

Use case: 12B fast 4-step generation model (Apache 2.0 license)

FLUX.2 [dev]

Source: black-forest-labs/FLUX.2-dev

Pipeline: Flux2Pipeline

Use case: 32B unified multi-modal model (text-to-image, inpainting, depth, canny, etc.)

---

Usage Examples

FLUX.1 Models (text-to-image, inpainting, depth, canny, etc.)

from diffusers import FluxPipeline  # or FluxFillPipeline, FluxControlPipeline, etc.
from diffusers.models import FluxTransformer2DModel
from transformers import T5EncoderModel
from optimum.quanto import QuantizedDiffusersModel, QuantizedTransformersModel
from huggingface_hub import snapshot_download
import torch

REPO_ID = "VincentGOURBIN/flux_qint_8bit"
quant_path = snapshot_download(REPO_ID)

# Quantized model classes for FLUX.1
class QuantizedFluxTransformer2DModel(QuantizedDiffusersModel):
    base_class = FluxTransformer2DModel

class QuantizedT5EncoderModel(QuantizedTransformersModel):
    auto_class = T5EncoderModel

# Example: Load FLUX.1-dev with quantized transformer
pipe = FluxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev",
    transformer=None,
    torch_dtype=torch.bfloat16
)

transformer = QuantizedFluxTransformer2DModel.from_pretrained(
    f"{quant_path}/flux-1-dev/transformer/qint8"
)
pipe.transformer = transformer.to("mps")  # or "cuda"

# Optional: Load quantized T5 text encoder (saves ~9GB)
# text_encoder_2 = QuantizedT5EncoderModel.from_pretrained(
#     f"{quant_path}/flux-1-dev/text_encoder/qint8"
# )
# pipe.text_encoder_2 = text_encoder_2.to("mps")

image = pipe("A majestic mountain at sunset", num_inference_steps=28).images[0]
image.save("output.png")

FLUX.2 Models (unified multi-modal)

from diffusers import Flux2Pipeline
from diffusers.models import Flux2Transformer2DModel
from transformers import AutoModel
from optimum.quanto import QuantizedDiffusersModel, QuantizedTransformersModel
from huggingface_hub import snapshot_download
import torch

REPO_ID = "VincentGOURBIN/flux_qint_8bit"
quant_path = snapshot_download(REPO_ID)

# Quantized model classes for FLUX.2
class QuantizedFlux2Transformer2DModel(QuantizedDiffusersModel):
    base_class = Flux2Transformer2DModel

class QuantizedFlux2TextEncoder(QuantizedTransformersModel):
    auto_class = AutoModel

# Load FLUX.2-dev with quantized transformer
pipe = Flux2Pipeline.from_pretrained(
    "black-forest-labs/FLUX.2-dev",
    transformer=None,
    torch_dtype=torch.bfloat16
)

transformer = QuantizedFlux2Transformer2DModel.from_pretrained(
    f"{quant_path}/flux-2-dev/transformer/qint8"
)
pipe.transformer = transformer.to("mps")  # or "cuda"

# Optional: Load quantized Mistral text encoder (saves ~36GB)
# text_encoder = QuantizedFlux2TextEncoder.from_pretrained(
#     f"{quant_path}/flux-2-dev/text_encoder/qint8"
# )
# pipe.text_encoder = text_encoder.to("mps")

image = pipe("A beautiful landscape", num_inference_steps=28, guidance_scale=4.0).images[0]
image.save("output.png")

---

Memory Requirements

Compatibility

Installation

pip install diffusers transformers accelerate safetensors
pip install optimum[quanto]
pip install huggingface_hub

Important Notes

• VAE is NOT quantized - Quantizing VAE causes visual artifacts
• LoRA compatible - Quantized models work with LoRA adapters (unlike GGUF)
• Text encoders are optional - Transformer-only quantization saves significant memory while the text encoder runs in bfloat16

File Structure

flux_qint_8bit/
├── flux-2-dev/           # FLUX.2 models (if present)
│   ├── transformer/
│   │   └── qint8/
│   └── text_encoder/
│       └── qint8/
├── flux-1-dev/           # FLUX.1 models
│   ├── transformer/
│   │   └── qint8/
│   └── text_encoder/
│       └── qint8/
├── flux-1-schnell/       # Fast model
│   └── ...
└── README.md

Generated With

flux-quantizer - Gradio tool for batch quantizing and publishing FLUX models.

---

Last updated: 2025-12-23 18:18 UTC