nocturne-v1.2-teacher

Nocturne v1.2 (Teacher) — non-bird bioacoustic species recognition

Nocturne identifies the taxa bird-focused models leave out: insects, amphibians, non-bird mammals and reptiles. v1.2 adds Neotropical anurans and, more importantly, fixes two defects in how v1 was supervised and selected.

v1.2 is the first model in this family to beat the v1 baseline on the frozen benchmark, on both metrics, measured on the identical 13,710-clip held-out test set with identical name-aligned class columns.

Both improvements exceed the noninferiority margin we pre-committed before scoring (0.00169 macro-F1, 0.00152 mAP), derived from a paired bootstrap over the test rows. They are improvements, not noise.

What actually produced the gain — and what did not

We ran four attempts at v1.2. Three failed to beat v1, and saying which is more useful than the headline:

The dataset was never the bottleneck. Two supervision bugs were:

1. One positive per training row. The loader emitted one row per (clip, species) pair with a single positive, so a clip containing two species was trained twice, each time teaching the model the other species was absent. Training now uses one multi-hot row per clip. This is the change that produced the gain.
2. Checkpoint selection by F1 at a fixed 0.5 threshold. The shipped metric is threshold-calibrated, so selection was optimising the wrong quantity. Selection is now by validation mAP. This model's best epoch was 18 of 40 — the old rule would have shipped a much later, worse checkpoint.

Where it does not improve

On a held-out monitoring site never seen in training (AnuraSet INCT41), v1.2 reaches top-1 0.138. That is roughly ten times better than the weak-label attempt (0.005), but still well below v1's 0.544. Adding AnuraSet has not bought generalisation to a new recording site. If your use case is deploying to an unseen site for anuran monitoring specifically, v1 may still serve you better, and we would rather say so than bury it. Only 3–5 species are scorable at that site, so treat that number as a signal, not a ranking.

Model

Training data

86,703 multi-hot training rows (one per audio file) from InsectSet459, the non-bird portion of iNat Sounds 2024, and AnuraSet strong labels (per-vocalisation time bounds, all call qualities, ≥50 % window overlap). Note the row count is files, not the source datasets' clip totals — an earlier version of this card conflated the two and overstated it.

Evaluation protocol

One frozen harness, soundscape/benchmark.py, scores every checkpoint identically: InsectSet459 + iNat held-out test, one multi-hot row per audio file, per-class thresholds calibrated on validation (grid 0.02→0.94, default 0.3 where a class has no validation positives), macro-averaged over the 2,182 v1 core classes matched by name so models with different vocabularies remain comparable. The harness is validated by reproducing v1's published numbers to four decimals before any comparison is trusted.

macro_f1 averages over all 2,182 columns, but only 732 have positives in the test split, so ~1,450 columns contribute exactly zero regardless of model quality. The companion figure over scorable classes only (0.4503) is the more interpretable one. Both are reported here; neither is cherry-picked.

Files

Usage

from huggingface_hub import snapshot_download
import sys

path = snapshot_download("stratus-labs/nocturne-v1.2-teacher")
sys.path.insert(0, path)
from model import load_nocturne, predict_file

model, vocab, thresholds = load_nocturne(path)      # strict — raises on any key mismatch
for species, score in predict_file(model, "clip.wav", vocab, top_k=5):
    print(f"{score:.3f}  {species}")

Never load these weights with strict=False. A silent partial load leaves the backbone at AudioSet initialisation and produces plausible-looking nonsense; that bug cost us a full round of invalid benchmarks.

Limitations

• Long-tail. 2,196 classes, 732 with test positives. Rare species are effectively unvalidated.
• Not for birds. Use BirdNET or Perch.
• New-site generalisation for anurans is worse than v1 (see above).
• Not for legal or conservation decisions without field verification.
• Dual-use. Acoustic localisation of rare taxa can aid illegal collection; consider redacting precise GPS in public datasets built with this model.

Related

• Baseline: Nocturne v1 teacher · distilled v1 mini
• Sibling family: Radiance (Sentinel-1 SAR)
• All releases: huggingface.co/stratus-labs

Research page

The write-up and every Nocturne release, in one place: https://runstratus.com/research/nocturne-v1-non-bird-bioacoustics