srt-cxr14-linear-probe

A 344 KB linear probe on frozen features, ahead of a fine-tuned ResNet-50

Linear(5376, 14) fitted on frozen google/gemma-4-31B-it hidden states. The backbone is untouched: no fine-tuning, no radiology pretraining, no augmentation. All 112,120 images of NIH ChestX-ray14, official test_list.txt.

Ahead on 12 of 14 findings. 14 of 14 clear the view-position baseline.

The whole result refits from the published states in 48 seconds on a MacBook CPU. No GPU is needed to reproduce it, only to encode the images once.

Read the split before comparing

CheXNet's 0.8414 and Yao's 0.8027 are on a different test set. CheXNet section 5: "We randomly split the dataset into training (70%), validation (10%), and test (20%) sets." That is their own random partition, not the official list.

Which split is harder is not established. Wang scored 0.7381 on a random partition and 0.7451 on the official one, so for their model the official split was marginally easier. The only supportable statement is that the two are not comparable, so exactly one split-matched row appears above.

Do not compare 0.7590 to 0.8414.

Per-finding

Intervals are a patient-level cluster bootstrap. The test split is 25,596 films from 2,797 patients, about 9 each, and those films are not independent. Resampling images instead of patients gives intervals roughly 1.5x too narrow.

The view-only baseline is folded (max(vw, 1-vw)). Hernia's raw view-only AUROC is 0.3436, which is 0.6564 of shortcut once flipped, and reporting the raw figure would have flattered the probe.

Usage

The probe expects a mean-pooled hidden state from the same backbone and layer, and the checkpoint carries the train-split mu and sd it needs. Applied to any other features, or without that normalisation, the scores mean nothing.

import torch
from huggingface_hub import hf_hub_download

ck = torch.load(hf_hub_download("RiverRider/srt-cxr14-linear-probe",
                                "cxr14_probe.pt"), weights_only=True)
x = (state - ck["mu"]) / ck["sd"]          # state: (n, 5376) from gemma-4-31B-it
p = torch.sigmoid(x @ ck["W"] + ck["b"])   # (n, 14), order in ck["findings"]

Precomputed states for all 112,120 images, the manifest with the official split, and the fitting script are in RiverRider/srt-cxr14-frozen-probe.

Scope, and what this is not

Detection, not early detection. These labels describe what is visible in the image in front of you. Nothing here speaks to catching disease before it is apparent, which needs longitudinal data with outcomes.

Not a diagnostic device. Research artifact. No clinical validation, no prospective evaluation, no regulatory clearance.

Labels are NLP-mined from radiology reports by the dataset authors. Every model on this benchmark inherits that ceiling.

One backbone, and the number depends on which. The probe is deliberately linear, because anything stronger starts measuring the probe rather than the representation. The identical probe, split and protocol on other frozen backbones gives Qwen3-Omni-30B-A3B 0.7650 and Aria 0.7080, so the 0.7590 here is a gemma-4 result and is labelled as one. Averaging three backbones' logits, which adds no parameters, reaches 0.7774: see RiverRider/srt-cxr14-pooled-probe.

Banked negatives

Part of the SRT program, https://github.com/space-bacon/SRT.