apcen-multihead-tagger

apcen-multihead-tagger — FSD50K (200 labels)

Efficient multi-label audio tagging with a learned, physically-grounded front-end (ERB SuperGaussian filter bank + adaptive PCEN / APCEN) and a three-head model on a shared encoder (peak-pool / modulation-spectrum / transformer-SED) fused by a per-class gate. 5.06M parameters, trained from scratch on FSD50K — no AudioSet pretraining.

Code: https://github.com/Thomas-Durand-Texte/apcen-multihead-tagger

Quick start

pip install git+https://github.com/Thomas-Durand-Texte/apcen-multihead-tagger

from apcen_multihead_tagger.model import ApcenMultiheadTagger

# No argument → downloads this checkpoint from the Hub (cached afterwards).
model = ApcenMultiheadTagger.from_pretrained()
probs = model.predict(waveform)      # [B, 200] sigmoid probabilities, mono @ 44.1 kHz
print(model.labels[:3])              # ['Accelerating_and_revving_and_vroom', 'Accordion', ...]

Or from the command line — --weights defaults to this repo:

python -m apcen_multihead_tagger.infer --input clip.wav --topk 5

Files

The checkpoint is a plain torch.save payload: model_state_dict (EMA weights), vocabulary (200 class names, index-aligned to the logits), and the selection metadata (epoch, val_lwlrap, val_mAP). Optimizer/scheduler state was stripped for release.

Model description

• Front-end: STFT → analytical ERB SuperGaussian filter bank (72 bands, 50 Hz–20 kHz) → APCEN (arXiv:2510.18206), a learned closed-loop per-band automatic gain control → learned 2D front-end.
• Encoder: factorized time-frequency (freq-first, frequency-dynamic conv, sub-spectral norm).
• Heads: three reducers with different temporal inductive biases (pool 86 fps / modulation-spectrum 43 fps / transformer-SED), fused by a per-class gate; each head adds an auxiliary loss that co-supervises the shared encoder.
• Output: [B, 200] logits → sigmoid → per-class probabilities.

Intended use

• Inference / tagging: predict FSD50K sound-event tags for audio clips.
• Transfer / fine-tuning: a compact backbone for your own labelled audio (see the repo's fine_tune.py; full fine-tune or --freeze-backbone feature-extraction).
• Feature extraction for downstream audio tasks.

Not intended for speaker ID, transcription, or any safety-critical use.

Metrics (FSD50K validation)

Checkpoint selection: best macro-mAP epoch (65), EMA weights. Macro mAP weights every class equally, so this selection favours the long tail; the lwlrap-best epoch (70) scores 0.7350 / 0.5992 — a 0.0002 lwlrap trade for +0.002 mAP.

This is the wc19 line. A larger successor (11.46M params, analytical SuperGaussian filter bank, transformer-only head) is still training and will be published separately — it does not supersede this checkpoint yet.

Per-head (single-head readouts on the same encoder): transformer 0.734 lwlrap / 0.598 mAP, pool 0.664 / 0.497, modulation 0.638 / 0.454. Honest note: at maturity the per-class gate largely concentrates on the transformer head — the pool/modulation heads' fusion contribution is small (~+0.003 mAP); their main value is the encoder co-supervision during training.

Training data

FSD50K (Fonseca et al., 2022) — 200 labels, ~41k clips, CC-BY 4.0. Weak (clip-level) labels, 44.1 kHz. No label corrections applied to this pretrain.

Limitations & biases

• Weak clip-level labels; no temporal event boundaries.
• Long-tailed label distribution — rare classes are harder.
• Trained on FSD50K's Freesound-sourced audio; domain shift (recording conditions, languages, unseen event types) will degrade performance.
• The FSD50K vocabulary is hierarchical (Purr and Animal can both fire) — treat the scores as a multi-label set, not a single argmax.

License & attribution

• Code: MIT.
• Weights / dataset: derived from FSD50K, released under CC-BY 4.0 — attribute FSD50K (Fonseca, Favory, Pons, Font, Serra, "FSD50K: an Open Dataset of Human-Labeled Sound Events", IEEE/ACM TASLP 2022) when redistributing.

Citation

If APCEN is central to your use, cite the adaptive-PCEN work (arXiv:2510.18206) and FSD50K.