flux.1-dev-modern-anime-lora

Modern Anime LoRA Adapter for FLUX.1 dev

Usage

• diffusers

1. Run the script:

import torch
from diffusers import FluxPipeline, FluxTransformer2DModel

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16)
pipeline.load_lora_weights('alfredplpl/flux.1-dev-modern-anime-lora', weight_name="modern-anime-lora_diffusers.safetensors")
pipe=pipe.to("cuda")

prompt = "modern anime style, An anime girl holding a sword. She is wearing school uniform. She is saying \"Hello World\" with the speech bubble."
image = pipe(
    prompt,
    height=1344,
    width=768,
    guidance_scale=3.5,
    output_type="pil",
    num_inference_steps=50,
    max_sequence_length=512,
    generator=torch.Generator(device="cuda").manual_seed(0),
    joint_attention_kwargs={"scale": 1.0}
).images[0]
image.save("flux-dev.png")

1. Get the following image:

• ComfyUI

1. Download v0.1 lora or v2.0 lora.
2. Move the file to ComfyUI/models/loras.
3. Lunch ComfyUI.
4. Load the workflow.
5. Queue prompt. (trigger words: modern anime style,)
6. Get the following image.

Examples

Please use ChatGPT or Claude to make a prompt!

v0.1

modern anime style, A close-up portrait of a young girl with green hair. Her hair is vibrant and shoulder-length, framing her face softly. She has large, expressive eyes that are slightly tilted upward, with a gentle and calm expression. Her facial features are delicate, with a small nose and soft lips. The background is simple, focusing attention on her face, with soft lighting that highlights her features. The overall style of the illustration is warm and inviting, with a soft color palette and a slightly dreamy atmosphere.

modern anime style, A full-body illustration of a girl with white curly hair styled in loose waves. She is wearing a black sleeveless top and blue hot pants. The girl has a mature and confident expression, with eyes that convey a sense of wisdom and allure. She is standing with a relaxed posture, perhaps with one hand on her hip, and the background is simple to keep the focus on her. The overall style of the illustration is elegant, with attention to detail in her clothing and features, and a modern, fashionable atmosphere.

v2.0

modern anime style, Anime girl with long purple hair. She is wearing a blue and yellow jacket with a white collared shirt underneath. The shirt has a red tie with a cross on it. The girl is making a peace sign with her hand.

How to make the LoRA Adapter

I used sd-scripts (the sd3 branch). The parameters is as follows:

accelerate launch --num_cpu_threads_per_process 1 flux_train_network.py --pretrained_model_name_or_path '/mnt/NVM/flux/flux1-dev.safetensors'  --clip_l '/mnt/NVM/flux/clip_l.safetensors'  --t5xxl '/mnt/NVM/flux/t5xxl_fp16.safetensors'  --ae '/mnt/NVM/flux/ae.safetensors'  --cache_latents --save_model_as safetensors --sdpa --persistent_data_loader_workers --max_data_loader_n_workers 2 --seed 42 --gradient_checkpointing --save_precision bf16 --network_module networks.lora_flux --network_dim 16 --network_alpha 16 --optimizer_type adamw8bit --learning_rate 1e-3 --network_train_unet_only --cache_text_encoder_outputs --cache_text_encoder_outputs --max_train_epochs 3 --save_every_n_epochs 1 --dataset_config flux_lora.toml --output_dir /mnt/NVM/flux --output_name flux_lora --timestep_sampling sigmoid --model_prediction_type raw --discrete_flow_shift 3.0 --guidance_scale 1.0 --loss_type l2 --mixed_precision bf16 --full_bf16 --max_bucket_reso 2048 --min_bucket_reso 512 --apply_t5_attn_mask --lr_scheduler cosine --lr_warmup_steps 10

[general]
enable_bucket = true

[[datasets]]
resolution = 1024 
batch_size = 4

  [[datasets.subsets]]
  image_dir = '/mnt/NVM/flux_lora'
  metadata_file = 'flux_lora.json'