SD3.5M-FlowGRPO-Text

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

This model is trained using Flow-GRPO with LoRA. We provide only the LoRA weights here, so you will need to download the SD 3.5 Medium base model first.

Model Details

Model Sources

<!-- Provide the basic links for the model. -->

• Repository: https://github.com/yifan123/flow_grpo
• Paper: https://www.arxiv.org/pdf/2505.05470

Uses

import torch
from diffusers import StableDiffusion3Pipeline
from diffusers.schedulers import FlowMatchEulerDiscreteScheduler
from peft import PeftModel

model_id = "stabilityai/stable-diffusion-3.5-medium"
lora_ckpt_path = "jieliu/SD3.5M-FlowGRPO-Text"
device = "cuda"


pipe = StableDiffusion3Pipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe.transformer = PeftModel.from_pretrained(pipe.transformer, lora_ckpt_path)
pipe.transformer = pipe.transformer.merge_and_unload()
pipe = pipe.to(device)

prompt = 'a weathered cave explorers journal page, with the phrase "lost city near" prominently written in faded ink, surrounded by sketches of ancient ruins and cryptic symbols, under a dim, mystical light.'
image = pipe(prompt, height=512, width=512, num_inference_steps=40,guidance_scale=4.5,negative_prompt="").images[0]  
image.save(f"flow_grpo_text.png")