SeeingTheUnseen

Seeing the Unseen — trained models

Detector weights and derived bases from a controlled study of occlusion compensation in aerial wildlife monitoring, along two axes: single view vs multi view, and sensor (pixel) space vs embedding space.

Code

https://github.com/Faline-eco/SeeingTheUnseen-Replication

The replication package: the pipeline that produced these checkpoints, the container it ran in, every scored metric behind the paper's tables, and a step-by-step in REPLICATION.md. Run directory names here are the run identifiers used there, so a checkpoint joins directly against metrics/*.json under the arms key.

Contents

Backbones are not included. DINOv3 and V-JEPA 2.1 are used frozen and unmodified — obtain them from their original sources. Only self-trained artefacts are here.

Nearly everything here is cross-validated. A run directory whose name carries _f<fold> belongs to the scene-level five-fold grid; 510 of the 534 run directories do. Superseded single-split runs have been removed.

Twenty-four single-split directories are deliberately kept, because two claims in the paper rest on them and nothing cross-validated replaces them:

• yolo/<arm>_s<seed> (12) — the ground-truth-choice comparison, which scores each detector against both the central and the merged annotation.
• heads/alfs_2k_*_2k_{64,128,256}_s<seed> (12) — the PCA-width ablation behind the choice of 128 dimensions.

The cross-validation grid in yolo/ is the complete {ortho, alfs} × {rgb, thermal} × 5 folds × 3 seeds = 60 runs behind the image-space result, each with its results.csv and the args.yaml it was trained with.

The learned-aggregator grid in heads/viewgrid_* is the complete {mean, attn, gru, swin} × {rgb, thermal} × 5 folds × 3 seeds = 120 runs behind the view-aggregation appendix, each with its summary.json (final metrics and configuration) and results.json (per-epoch log). best.pt carries agg, hid and win, and scripts/view_aggregator.py::ViewAggregatorDetector.from_checkpoint in the replication package rebuilds the model. These checkpoints read the registered view stack (the per-view DINOv3 fields warped onto the grid on the fly from the sampling geometry that render_embedding_field.py --emit grid writes), not the integrated cell the other heads read.

Fine-tuned OWL-D checkpoints are not published. The metrics are in the replication repository (metrics/owlft_rgb_f*.json) and the procedure in docs/OWL_BASELINE.md, but each checkpoint carries a full DINOv3 backbone at 3.3 GB and the ten of them were judged not worth 33 GB here.

Naming

Directory names are the run identifiers used throughout the study, so they join directly against the logged metrics in the replication repository (metrics/*.json, under the arms key):

heads/cell_thermal_embed_multi_embed_multi_f0_s1337/
      cell_<modality>_<cell>_<label>_f<fold>_s<seed>

heads/viewgrid_thermal_embed_multi_swin_f0_s1337/
      viewgrid_<modality>_embed_multi_<aggregator>_f<fold>_s<seed>

yolo/ortho_thermal_f0_s1337/
      <aperture>_<modality>_f<fold>_s<seed>

A name without _f<fold> is one of the 24 retained single-split runs described above, not a fold of the cross-validation.

_debias32 marks arms with the INSID3 positional subspace projected out. Cell families: embed_* (DINOv3, encode-then-integrate), vjepa_* (V-JEPA 2.1), alfsembed_* (integrate-then-encode), and sensor_*, realalfs_*, realortho_* (pixel space). Aggregators: mean (the control, which reproduces the embed_multi cell), attn (masked softmax over learned per-view, per-cell scores, 27 k params), gru (bidirectional GRU over the views in flight order, 75 k), swin (two windowed self-attention blocks over 4×4 cells × all views, 90 k); each starts as the mean at initialisation.

Protocol

Scene-level 5-fold cross-validation over 72 flights, balanced on hidden-box count, 3 seeds per fold. No frame from a validation flight appears in training.

Each fold and seed has its own checkpoint. Pick the checkpoint by fold and never pool across folds, or the held-out guarantee is lost.

• Heads: AdamW 3e-4, cosine schedule, batch 16, up to 60 epochs (patience 12), focal + 5x L1 loss.
• Learned aggregators: as the heads, up to 40 epochs (patience 12), fp16 autocast, checkpoint chosen on the central-frame ground truth.
• YOLO: 1024 px, batch 4, up to 100 epochs (patience 10).

Licence

MIT for heads/, pca/ and bases/ — see LICENSE in the replication repository, which covers the code as well.

AGPL-3.0 for yolo/ only. Those weights are fine-tuned from Ultralytics YOLO26x and inherit its terms. AGPL-3.0 applies if you use those checkpoints; it does not reach anything else here.

The repository is declared other rather than mit because the two cannot be expressed in one field, and labelling the whole thing MIT would misrepresent the YOLO weights.

Imagery derives from the public BAMBI dataset under its own terms. DINOv3 and V-JEPA 2.1 are used frozen and unmodified and are not redistributed here; obtain them from their original sources under their own licences.