garment-attributes-v2

garment-attributes-v2

Same architecture, same label space, same hyperparameters as resoa/garment-attributes — retrained with crop-scale augmentation. +11.4% relative macro-mAP.

218 Fashionpedia construction attributes from a garment crop. SigLIP2-base vision encoder with a classification head, 93.1M parameters.

What changed, and what actually caused the gain

v1 trained on crops padded 8% around the bounding box. v2 trains at 25% padding with ±10% random jitter, plus horizontal flip. Nothing else differs — same optimiser, LR, schedule, batch size, epoch count and label space.

Both changes were ablated separately, because the experiment was designed around padding and it turned out not to be the main effect (Fashionpedia val2020, 3,711 instances, bootstrap 400 resamples):

Scale jitter contributed 61% of the improvement and the padding value only 39% — so this model is best understood as v1 trained with crop-scale augmentation, not as v1 trained at better padding. The mechanism is visible in the per-padding table below: a model trained at a fixed 0.25 scores worse than v1 when evaluated at 0.08. Fixing the training scale buys accuracy at that scale and loses it elsewhere; jitter is what makes the model robust to how tightly you crop.

Results — Fashionpedia val2020, the parent's own protocol

3,711 instances, 212 evaluable labels, 48-pixel minimum crop.

Best-to-best: +0.0492 ± 0.0062, P(v2 better) = 1.00. micro-F1 0.7006 → 0.7244. v2 wins at every padding, so it has not merely shifted its preferred operating point.

The gain lands where v1 was weakest

172 of 212 labels improved, 40 regressed. Grouped by training support:

The improvement is inversely proportional to how much data an attribute had — augmentation helps most exactly where examples are scarcest. Largest gains: sleeveless 0.267→0.833, teddy bear (coat) 0.279→0.722, raglan (t-shirt) 0.093→0.385.

Regressions are real and reported. tuxedo (jacket) fell 1.000→0.200, though with very few val positives its AP is unstable in both directions; collarless 0.688→0.511, banded (collar) 0.307→0.198. Per-attribute v1/v2 numbers ship in per_attribute_v2.json so you can check the attributes you care about rather than trusting the mean.

Measurement honesty

This harness reproduces the parent's protocol exactly — 3,711 instances and 212 labels, matching the published counts — but scores v1 at 0.4229 macro-mAP against the published 0.4417, a ~2% shortfall most likely from crop-rounding or resize detail. Every comparison above is v1 and v2 measured by the same harness, so that offset cancels. Read the absolute numbers as ~2% low and the deltas as sound.

Usage

Identical to v1 — drop-in replacement.

from transformers import AutoImageProcessor, AutoModelForImageClassification
import torch
from PIL import Image

model = AutoModelForImageClassification.from_pretrained("resoa/garment-attributes-v2")
proc = AutoImageProcessor.from_pretrained("resoa/garment-attributes-v2")
img = Image.open("garment_crop.jpg")          # crop of ONE garment or garment part
probs = torch.sigmoid(model(**proc(images=img, return_tensors="pt")).logits)[0]
for i, p in enumerate(probs):
    if p > 0.5:
        print(model.config.id2label[i], round(p.item(), 3))

Crop at ~25% padding around the garment for best results. v2 is markedly more forgiving of looser crops than v1 (0.4647 vs 0.3842 at 0.50 padding), which is the practical benefit of the jitter.

Inherited limitations

Everything true of v1 remains true, and the training change fixes none of it:

• Not calibrated. Sigmoid outputs are not probabilities.
• Crops, not scenes. A full scene retains only ~14% of trained-crop mAP. See resoa/garment-crop-gate-nano, a 47K-param gate that declines inputs too loosely cropped to trust.
• Worn-garment street photography. Flat-lay and factory photos are out of domain.
• The label space spans garments AND garment parts. Fashionpedia annotates sleeves, pockets, collars and lapels as separate instances — a welt (pocket) label was learned from crops of pockets at ~0.3% of the source frame, not from whole garments containing pockets. nickname attributes never co-occur (mean 1.012 per instance) for the same reason.
• Cannot predict fibre content, GSM, measurements or colour.

Training

156,937 crops from Fashionpedia train2020 (≥100 training instances per attribute, 48-pixel minimum crop), 4 epochs, batch 32, AdamW lr 2e-5, cosine schedule, bf16. 1.68 hours on a Jetson AGX Orin.

Does it beat a trivial baseline?

Every nano model in this line is required to beat the best of ten cheap image statistics before it ships — a rule that has killed several candidates. That rule was never applied to this 93M model until a card audit noticed the omission. Measured on a held-out half of Fashionpedia val2020, 204 evaluable attributes:

Margin +0.4386 — 7.6× the cheap baseline. The model earns its parameters, which is not something to assume: three candidate models in this programme lost outright to a single threshold.

Known failure modes, measured

• Small features are not resolvable. 19 of 202 attributes have a diagnostic feature under 2 pixels at this input size; metal (5 mm) scores AP 0.072 despite 1,135 training examples, and bead(a) 0.133 despite 2,030. No amount of data fixes a one-pixel feature.
• Rare attributes are weak. Mean AP 0.358 for attributes with 100–300 training instances against 0.770 above 5,000.
• Loose crops collapse it. A full scene retains only 14% of trained-crop mAP.
• Regressions from v1 exist and are shipped in per_attribute_v2.json — tuxedo (jacket) 1.000 → 0.200, collarless 0.688 → 0.511.