toy-diffusion

Toy-diffusion Ablation Results & Benchmark Analysis

Following the results presented in the i1 paper we conducted a series of architectural controlled experiments on our aipracticecafe/anime-faces-256px-v2 dataset (comprising 87,505 high-quality face crops) using Flux 2 VAE latents and using the Toy-diffusion training code.

Both Single and Dual stream dits completely remove the ADA-LN conditioning, instead the timestep is projected as a token and prepended to the text tokens. Training uses AdamW 8bit optimizer with warmup-constant learning rate of 4e-4 and a batch size of 256.

Below is the detailed benchmark performance FID-20K (Fréchet Inception Distance, ) across our evaluated checkpoints.

1. Unified Leaderboard of Evaluated Checkpoints

---

Architectural Insights

Text adapter

Our results provide clear, empirical validation of the importantece of a strong text encoder adapter:

• Comparing dual_stream_768_16_mlp (depth 16, FID 4.57 at 500 epochs) to dual_stream_768_12_trans_2 (depth 12, FID 3.54 at 500 epochs) reveals a clear pattern. Investing capacity into a 2-layer Transformer Text Adapter (trans_2) rather than raw MLP produces a massive visual upgrade.

B. Single-Stream vs. Dual-Stream Backbones

• Training Efficiency: Dual-stream blocks completed training epochs in ~3.5 minutes, whereas Single-stream blocks required ~5.1 minutes—under identical VRAM constraints (~23GB). By processing text and image tokens independently, Dual-stream models seem to be more hardware-friendly.
• Visual Fidelity: Dual-stream models drop FID significantly faster and settle at a much lower ultimate floor (FID 3.06 vs. Single-stream's 3.69 plateau).
• The Overfitting & Cosplay Tradeoff: We observed a distinct qualitative divergence:
  • Dual-stream models are prone to localized concept overfitting (e.g., struggling to render Asuka in Misato's uniform, consistently over-associating characters with their native outfits).
  • Single-stream models handle complex conditional blending and "cosplay" prompts with far better zero-shot composition, indicating stronger cross-modality token integration.

C. SPRINT: Efficient Sparse Pre-Training

• The sparse token-dropping framework (SPRINT) with a 75% drop ratio achieved a ~50% reduction in training time and lowered active GPU memory requirements from 23GB to 17GB. Even with the Finetuning stage, the overall training wall time is shorter than the dense version.
• Finetuning recovers performance: Evaluating SPRINT models directly results in highly degraded FID scores (FID ~13.8) due to the dense-sparse distribution shift. However, following the dense finetuning recipe (_fined), the model recovers rapidly, achieving an FID of 3.96 by epoch 1000.
• Dual Stream is favored: Despite the good FID scores exhibited by the Single stream architecture, during the finetuning stage only the dual stream architecture is able to recover the high performance.

Developed by: aipracticecafe

Content Warning & Rights

• NSFW Content: This dataset contains sensitive and NSFW (Not Safe For Work) material. It is intended for research and generative modeling purposes.
• Legal Disclaimer: I do not own the rights to any of the images in this dataset. All images are the property of their respective creators and were scraped from Danbooru. This dataset is provided for educational and research purposes under fair use.