clothing-segmentation-segformer-b0

Clothing Segmentation — SegFormer-B0

Binary clothing segmentation model fine-tuned on the iMaterialist Fashion 2020 (FGVC7) dataset. Separates worn clothing from images of people at the pixel level — built for use cases like virtual fitting rooms, fashion cataloging, and outfit-swap applications.

Full training code, technical report, and documentation: GitHub repository

Performance

Model Details

• Architecture: SegFormer-B0 (nvidia/segformer-b0-finetuned-ade-512-512), fine-tuned with a binary (1-channel) segmentation head
• Task framing: Binary segmentation (clothing vs. background) rather than multi-class, to match real-world virtual-fitting-room use cases
• Loss function: Combined Dice + Binary Cross-Entropy (BCE) loss, to handle background-pixel-dominant masks
• Input resolution: 512×512
• Training data: 3,000-image subset of iMaterialist Fashion 2020 (FGVC7), 85/15 train/validation split
• Training hardware: Kaggle free-tier T4 GPU, mixed precision training

Available Checkpoints

Usage

Download the checkpoint:

wget https://huggingface.co/UseItOrLoseIt/clothing-segmentation-segformer-b0/resolve/main/best_model_inference_only.pth -O checkpoints/best_model.pth

Then, using the inference code from the GitHub repo:

python inference.py --image_path your_photo.jpg --output_path result.png

Limitations

• Performance degrades in low-light or backlit scenes
• Under-segments non-standard garment shapes (e.g., flared or trailing hemlines)
• Minor boundary imprecision at low-contrast garment-to-garment transitions (e.g., dark pants meeting dark boots)
• Edge sharpness is good but not pixel-perfect, a tradeoff of SegFormer-B0's lightweight decoder

Full analysis with annotated visual examples: report.md

Citation

If you use this model, please reference the GitHub repository.