duo-image-best

Duo vs. No‑Duo — Best Fine‑Tuned Model

Task: Binary image classification Dataset: scottymcgee/duo-image-dataset Backbone (selected via a small AutoML sweep): google/vit-base-patch16-224 Input resolution: 224×224

Expected preprocessing

• Resize shortest side to 224, center‑crop to 224×224
• Normalize with the backbone's mean/std (see preprocessor_config.json)

Augmentation pipeline (train)

• RandomResizedCrop, HorizontalFlip, ShiftScaleRotate, ColorJitter, Perspective, GaussianBlur, Normalize

Validation/Test protocol

• Train/val on the augmented split; test strictly on the originals to avoid augmentation leakage.

Training curves & early stopping

Early stopping was used (patience=2). Best eval_accuracy = 1.0000 at epoch 5; training stopped at epoch 5 after 0 non-improvement evals. Best checkpoint: runs/vit-base-patch16-224/run-0/checkpoint-40.

Test metrics (originals only)

Accuracy: 1.000 • Macro F1: 1.000 • Weighted F1: 1.000

Confusion matrix

Known failure modes

• Sensitive to out‑of‑distribution backgrounds and lighting.
• Small or occluded subjects lower accuracy; edge-cropped subjects can flip predictions.
• Non‑RGB or images not preprocessed to 224×224 + the stated normalization degrade performance.
• Heavily filtered/edited images (cartoonized, extreme color grading) diverge from the train distribution.

Reproducibility & budget

• Fixed seed = 42; early stopping enabled; small HPO sweep across backbones and core hyperparameters.
• Budget ≈ 30 GPU‑minutes on a single T4.

Purpose

Classify images into duo vs no_duo to demonstrate a compact, reproducible fine‑tuning and AutoML workflow for small image datasets (course project / teaching demo).

Data origin & splits

Origin: scottymcgee/duo-image-dataset (peer‑provided). Splits used: All original images reserved for test; augmented split used for train/val with a stratified 85/15 split. No augmented views of test originals are used at train/val (leakage guard).

Features & targets

Features: RGB images, expected 224×224 after preprocessing (resize + center crop + normalize). Target labels: duo, no_duo.

Training setup

Backbone: google/vit-base-patch16-224 (best). Optimizer: AdamW (Transformers default). Batch sizes: train 16 or 32, eval 64. Scheduler: cosine or linear (selected by HPO). Epochs & early stopping: early stopping with patience=2; best epoch and checkpoint recorded on the card.

Hyperparameter search space

We performed a small budgeted HPO sweep across four backbones (ViT‑B/16, ConvNeXt‑T, ResNet‑50, Swin‑T) and tuned:

• learning_rate: 5e‑6 → 5e‑4 (log‑uniform)
• weight_decay: 1e‑6 → 5e‑3 (log‑uniform)
• warmup_ratio: 0.0 → 0.1
• per_device_train_batch_size: 16 or 32
• lr_scheduler_type: cosine or linear
• label_smoothing_factor: 0.0 → 0.2

Limitations & ethical considerations

• Very small test set; metrics can be optimistic and have high variance.
• Labels are simple and may not capture nuanced real‑world definitions of “duo.”
• Model may learn spurious context (backgrounds, attire) not causally related to the concept.
• Do not use for safety‑critical or identity‑sensitive decisions.

AI usage disclosure

• Pretrained model (google/vit-base-patch16-224) fine‑tuned with Hugging Face Transformers.
• AutoML/HPO performed with a small Optuna sweep under a stated GPU‑budget.
• No synthetic data or generated labels were used; training data came from the cited dataset.
• Some documentation/code scaffolding assisted by generative AI (ChatGPT); human‑reviewed.

License

MIT (see repository header).