paper-to-sub-field-distilled

paper-to-sub-field-distilled

Static (CPU-speed) classifier that maps a scientific paper's title + abstract to the OpenAlex taxonomy at the subfield level, and derives field and domain from it.

Returns per paper: top-3 subfields, top-2 fields, 1 domain.

Why subfield-level

paper-to-field predicts the field with a 26-way head and then resolves the subfield with a separate FAISS nearest-topic lookup. Two independent decisions that can disagree -- and did, for 37.5% of a 116M-paper corpus. The FAISS step also collapses onto a few attractor centroids (Dermatology absorbed 3.4M papers while Behavioral Neuroscience got 29).

This model has a single head over subfields. Field probability is the sum of its subfields' probabilities; domain is a lookup from the top field. Nothing is predicted twice, so the returned hierarchy cannot be internally inconsistent.

Performance

Held-out test set (9,026 papers, 252 subfields):

Field top-1 is on par with the BioM-ELECTRA teacher it derives from (86.3%), at CPU speed and ~1/40th the parameters. Trained in 5.54 minutes on one GPU.

Macro (mean per-class recall, every subfield weighted equally): subfield top-1 0.0%, top-3 0.0%, 0 subfields with zero recall.

Filtering

Four gates run before training: abstract length >= 300 chars, English only (langdetect >= 0.90 on the truncated text the model sees), the label-agreement arm below, and optional per-class capping.

Per-class capping was tested and rejected. Capping each subfield at 400 rows raises macro subfield top-1 by 3.2 points and revives 4 dead classes, but costs 7.2 points of micro subfield accuracy and 1.5 of field. For per-paper placement against the natural distribution of incoming papers, the uncapped model is the better instrument; if you care mainly about rare-subfield recall, retrain with --per-class-cap 400.

Training data and its limits

88,485 paper abstracts. Labels come from OpenAlex primary_topic -> subfield. OpenAlex field labels are noisy: on this corpus they agree with a DeepSeek re-annotation on only 50.4% of rows, consistent with the ~50% error rate paper-to-field v3 was built to correct.

Arm hybrid: the agreement filter, plus the non-agreeing rows of any subfield left under 5 agreeing examples. Keeps verified labels where they are plentiful and keeps coverage where they are not, so all 252 subfields stay expressible -- including Drug Discovery, one of the six subfields defining the Cancer cross-discipline field. Only 13 classes are rescued this way; raising the floor to 40 rescues 49 and costs 2.1 points of field accuracy.

Training on all 199,293 rows instead is strictly worse, not merely noisier -- field accuracy falls to 59.7% and domain to 77.2%. Twice the data costs 27 points when half of it carries a disputed field, because the model learns the contradiction. Do not "improve" this by adding the filtered-out rows back.

Known limitations:

• Subfield labels are OpenAlex-derived and were never independently adjudicated. Subfield accuracy here measures agreement with OpenAlex, not correctness. Treat it as "gospel enough", not ground truth.
• OpenAlex records are frequently incomplete; the labels carry that noise.
• 0 subfields are absent from the head (full 252-subfield coverage).
• Text is truncated to title + the first 500 characters of the abstract, matching training.
• Subfield accuracy is not comparable across fields: it scales with how many subfields a field has (Veterinary has 2, Medicine 42).

Usage

from subfield_classifier import SubfieldClassifier

clf = SubfieldClassifier()
clf.classify(title="Attention Is All You Need", abstract="The dominant sequence ...")
# {'subfields': [{'id': 1702, 'name': 'Artificial Intelligence', 'field': 'Computer Science',
#                 'score': 0.87}, ...],
#  'fields': [{'name': 'Computer Science', 'score': 0.9998}, ...],
#  'domain': 'Physical Sciences'}