ours-probe-coco-llava15-13b

Ours-Probe — LLaVA-1.5-13B, COCO

Object-hallucination probe trained on a LLaVA-1.5-13B COCO corpus, with three independent training seeds (0, 42, 1337). It is not interchangeable with the 7B release: features, geometry and thresholds are model-specific.

Corpus

10,000 COCO train2014 images split 80/20 by image: 8,000 train / 54,253 object spans and 2,000 holdout / 13,789 object spans. Captions are generated by 13B itself, so this corpus is not the 7B corpus.

Results (mean +/- SD over the three seeds)

Thresholds are selected by best F1 on the COCO validation split and then frozen. Nothing is tuned on the test holdout, AMBER, or POPE.

COCO test holdout — mention view, n=13,787, 16.19% positive

AMBER, strict zero-shot — n=2,367 rows / 977 images, 5.37% positive, COCO thresholds frozen

AMBER's object universe is the human-labelled (image, noun) set; nouns 13B mentions that carry no AMBER label are not scored (2,367 of 3,350 candidate pairs matched).

POPE-Adversarial, zero-shot — n=3,000, 18.9% positive

Validation best-F1 threshold, seeds 0 / 42 / 1337: 0.59176 / 0.66603 / 0.72769

Layout

checkpoints/              one per seed
cache/coco/{train,holdout}/   features over the 13B corpus
cache/amber/seed*/        zero-shot AMBER features
cache/pope_adversarial/   zero-shot POPE-Adversarial features
metadata/model_profile.json   model id, layers, width, heads, vision grid
metadata/thresholds.json      validation-selected thresholds per seed
metadata/manifest.json        corpus provenance and cache source

Before using these artifacts, compare metadata/model_profile.json with model_workflow.sh show-profile; a mismatch means the cache must be rebuilt.

Which head to use

Two heads are fitted from one cache. probe_verify_raw_* uses h(image); probe_verify_contrast_* uses h(image) - h(no image). Every number above is the raw head — metadata/thresholds.json records ours_mode. Pass the matching mode at scoring time, or the thresholds do not apply.