danbooru-tagger-v1

danbooru-tagger-v1

An EVA02-L Danbooru tagger with a 24,000-tag general vocabulary, fine-tuned including the encoder from wd-eva02-large-tagger-v3 on 677,355 posts newer than any public tagger's training cutoff.

Every prior WD-derived model froze the encoder. pixai-tagger-v0.9 says so on its own card — "We continued training the classification head of EVA02 (from WD v3) … encoder frozen" — and it scores below the plain WD model it started from. Unfreezing is what this release tests. Internal validation macro-F1 went 0.4380 → 0.5692 when the encoder was unfrozen.

Read the results section before using this. v1 wins some metrics and loses one, and which you care about depends on what you are building.

Results

11,639 held-out posts with id > 8,600,750 — after the training cutoff of every model compared (WD 7.22M, Camie ~8.36M, PixAI 8.60M), so none of them has seen these images. Scored on the intersection of all seven vocabularies (7,779 tags; 3,101 with ≥10 positives), because macro-F1 over each model's own vocabulary rewards having a bigger vocabulary rather than being more accurate.

coverage = the share of tags actually present in the eval set that the model is able to emit at all.

The verdict is split, and it is not a clean win.

• macro-F1 +0.0161 and micro-F1 +0.0200 over the best incumbent — won.
• coverage 0.914 vs 0.506 — won, and by a lot.
• macro-AP 0.4672 vs 0.4668 — a tie. On the tags both models share, ranking quality is identical; the F1 win comes from calibration, not discrimination.
• fine-grained AP 0.5643 vs 0.5979 — lost, and it is not noise: bootstrap over images gives −0.0336, 95% CI [−0.052, −0.015].

Per-tag, the pattern is consistent: v1 improves where WD is weak and degrades where WD is strong. It gains nontraditional_miko +0.387 (WD manages only 0.186), cityscape +0.097, hands_up +0.081, miko +0.070; it loses lily_(flower) −0.182, flower_field −0.168, v −0.142, bouquet −0.125, torii −0.116. That is drift from a single-epoch fine-tune, not a worse encoder.

The configuration that does beat WD: ensemble.py

Because v1 and WD fail on different tags (probability correlation r = 0.876), averaging them beats either alone. Paired bootstrap over images, vs wd-eva02:

Ensembling anything helps — that control row matters. But v1 + wd-eva02, two models, beats an ensemble of all five incumbents (macro-AP 0.4919), and adding v1 to the three-WD ensemble lifts macro-AP by a further +0.029. On macro-AP, v1 is the most valuable single member available. On fine AP its marginal contribution over a WD-only ensemble is smaller (+0.0117) but still positive.

ensemble.py implements the two-model blend: average on the 7,800 shared general tags, v1 alone on the other 16,200. It costs two forward passes.

Which should you use?

• Deployed tagger with a threshold, or you need the 16,199 extra tags → v1 alone (predict.py). Better F1, far better coverage, names the specific tag more often (79.8% vs 72.3% on fine/parent pairs).
• Ranking quality matters most and you can afford 2× inference → ensemble.py.
• You only care about WD's 8,106 tags and want one model → keep using wd-eva02-large-v3. v1 does not beat it on fine-grained AP.

Usage

pip install onnxruntime pillow numpy
python predict.py image.png --thr 0.38

# the blend that wins on ranking; downloads wd-eva02 on first run
pip install torch timm pillow numpy huggingface_hub
python ensemble.py image.png --thr 0.38

Thresholds: 0.38 maximises macro-F1, 0.73 maximises micro-F1. Both were swept on the evaluation set itself, so treat them as reasonable defaults rather than as held-out-validated values — the same sweep was applied to every model in the table, so the comparison is fair even though the absolute F1s are optimistic.

Preprocessing — this is not optional

The model was trained on a specific pipeline and degrades quietly if you change it:

1. composite alpha onto white
2. pad to a square with white
3. bicubic resize to 448×448
4. scale to [0,1], then (x - 0.5) / 0.5
5. swap RGB → BGR
6. transpose to NCHW

timm's resolve_data_config returns CLIP normalisation for this architecture, which is wrong here. predict.py is the reference implementation.

Files

Not a drop-in replacement for WD's ONNX. WD publishes [batch,448,448,3] NHWC with 10,861 outputs; this is NCHW with 24,000 outputs and a different tag list. Existing WD14 tooling needs a transpose and the tag CSV from this repo.

Caveats

61% of the vocabulary is untested. Of 24,000 tags, 14,626 have fewer than 1,000 training examples, and on an 11,639-image eval set almost none appear often enough to score:

The tail is not shown to be good or bad — it is unmeasured. "24,000 tags" rests on roughly 5,173 tags this eval set can actually check. selected_tags.csv carries n_train per tag so you can see what each prediction is backed by.

Other limitations:

• Trained 1 epoch, not 2 — the unfrozen encoder ran at 43.8 img/s against 120 img/s frozen, and 2 epochs did not fit the compute budget.
• The training corpus was stored at 640px long side to fit on disk, so the model only ever saw double-resampled images (native → 640 → 448). It is scored that way too rather than tuned around.
• Ratings and character tags are not included; general tags only.
• fine AP above averages only 31 tags across 3 groups, so its confidence interval is wide (±0.019). Treat small differences on that column with care.

Training

• Base SmilingWolf/wd-eva02-large-tagger-v3, encoder unfrozen after a head-only warm-up of 0.3 epoch.
• 677,355 images: 548,410 posts newer than 8,600,750 plus 126,704 replay images from older posts, built from nyanko-devs/danbooru2026.
• Head extended to 24,000 general tags, overlapping rows warm-started from the base. New rows start at zero weight with a log-odds bias from the tag's base rate — initialising them the usual way made 16,199 tags emit sigmoid(0)=0.5 and outrank every real prediction, dropping precision@20 from 0.762 to 0.037.
• Asymmetric Loss (γ⁻=4, γ⁺=1, clip 0.05); instance-aware repeat-factor sampling (thresh 1e-3, cap 4, mean repeat 2.32×); discriminative LRs, encoder 1e-5 and head 1e-3, cosine to 5%; bf16; ~100.5k steps at batch 16×2.
• Split rule: everything except post_id > 8,600,750 AND id % 1000 >= 950, which is provably disjoint from the evaluation set.

License

Apache-2.0, inherited from wd-eva02-large-tagger-v3. Tag data originates from Danbooru.