Downloads · 30 days
40
18% of all-time downloads
je-suis-tm/wong_kar_wai_fallen_angels_lora_flux2_nf4
wong_kar_wai_fallen_angels_lora_flux2_nf4 is a text-to-image model from je-suis-tm. Use it when you need an image from a text prompt. It is set up for diffusers. The card lists the license as mit.
Downloads · 30 days
40
18% of all-time downloads
All-time downloads
219
Public
Repo size
27.8 MB
Likes
3
Public
Click a slice to open those files.
.png21.5 MB · 77%
From the Hugging Face model README
All files are also archived in https://github.com/je-suis-tm/huggingface-archive in case this gets censored.
Wong Kar Wai is one of my fav directors. This QLoRA intends to replicate that bold neon color tone, extreme wide angle, surreal neo-noir vibe of the movie Fallen Angels (1995). The comparison between flux1 https://huggingface.co/je-suis-tm/wong_kar_wai_fallen_angels_lora_flux and flux2 is very obvious. Flux1 does not support different aspect ratios in training but it does not distort human faces or female bodies in training dataset. Flux2 can capture the ultra wide angle lens, camera angles and lightings, but it really fucks up the faces and bodies and does not disclose anything about the shady operation which is fucking disgusting. It is up to you to decide which lora reflects the aesthetics of Wong Kar Wai. If flux3 behaves like this, I am switching to wan or qwen, fuck that censorship.
The training is based on https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/test_dreambooth_lora_flux2.py. Everything in this training script needs to be set at the same torch dtype as the script was designed for unquantized version which will export in float32. The training took 11 hours on A100 80GB with max VRAM consumption at 32GB. The inference consumes 35GB of VRAM. To avoid running low on VRAM, both transformers and text_encoder were quantized.
export MODEL_NAME="diffusers/FLUX.2-dev-bnb-4bit"
export INSTANCE_DIR="/pvol/wong_kar_wai_fallen_angels"
export OUTPUT_DIR="/pvol/wong_kar_wai_fallen_angels_lora_flux2_nf4"
export Q_DIR="/pvol/quantization_config.json"
accelerate config default
accelerate launch train_dreambooth_lora_flux2.py \
--pretrained_model_name_or_path=$MODEL_NAME \
--mixed_precision="bf16" \
--dataset_name=$INSTANCE_DIR \
--output_dir=$OUTPUT_DIR \
--bnb_quantization_config_path=$Q_DIR \
--gradient_checkpointing \
--cache_latents \
--instance_prompt="WKW style" \
--caption_column="text" \
--aspect_ratio_buckets="800,1456" \
--center_crop \
--train_batch_size=1 \
--guidance_scale=1 \
--use_8bit_adam \
--offload \
--checkpointing_steps=100 \
--gradient_accumulation_steps=4 \
--optimizer="adamW" \
--learning_rate=1e-4 \
--lr_scheduler="constant" \
--lr_warmup_steps=100 \
--max_train_steps=1500 \
--rank=4 \
--seed="0"
import torch
from transformers import Mistral3ForConditionalGeneration
from diffusers import Flux2Pipeline, Flux2Transformer2DModel
repo_id = "diffusers/FLUX.2-dev-bnb-4bit"
device = "cuda:0"
torch_dtype = torch.float32 #only supports float32 when using train_dreambooth_lora_flux2.py
transformer = Flux2Transformer2DModel.from_pretrained(
repo_id, subfolder="transformer", torch_dtype=torch_dtype, device_map="cuda:0"
)
text_encoder = Mistral3ForConditionalGeneration.from_pretrained(
repo_id, subfolder="text_encoder", dtype=torch_dtype, device_map="cuda:0"
)
pipe = Flux2Pipeline.from_pretrained(
repo_id, transformer=transformer, text_encoder=text_encoder, torch_dtype=torch_dtype
)
pipe.load_lora_weights("je-suis-tm/wong_kar_wai_fallen_angels_lora_flux2_nf4",
weight_name='pytorch_lora_weights.safetensors')
pipe.enable_model_cpu_offload()
prompt = "WKW Style, The image shows a man reclining on his back on top of a counter with a cigarette in hand. He's dressed casually in a loose white shirt and khaki pants, his head tilted upwards. Empty sauce bottles line the counter beside him, arranged neatly. The scene conveys a sense of repose, suggesting a moment of rest or a break, perhaps in a relaxed setting like a cafe. His body language is relaxed, hinting at a quiet, contemplative mood"
image = pipe(
prompt=prompt,
generator=torch.Generator(device=device).manual_seed(42),
num_inference_steps=50, # 28 is a good trade-off
guidance_scale=4,
height=800,
width=1456, #the movie was in a weird resolution 1056*1920, the resolution needs to be divided by 16
).images[0]
image.save("wong_kar_wai_fallen_angels.png")
You should use WKW style to trigger the image generation.
Download them in the Files & versions tab.