paper-to-sub-field-adjudicated

paper-to-sub-field-adjudicated

Static (CPU-speed) classifier mapping a scientific paper's title + abstract to the OpenAlex taxonomy at the subfield level, with field and domain derived from it. Returns top-3 subfields, top-2 fields, 1 domain.

Sibling of paper-to-sub-field-distilled. Same architecture, same recipe, same base model. The only difference is the training labels, and that difference is the entire point.

Why it exists

The sibling model was trained on OpenAlex's own primary_topic -> subfield labels. Those labels are unreliable. Relabelling 175,906 papers with an independent model (DeepSeek V4-Flash) found that OpenAlex's subfield is not the independent first choice for 58.9% of papers, and absent from its top three entirely for 29.9%.

The disagreement is concentrated rather than uniform -- Biochemistry/Genetics/Molecular Biology agrees 22.0% and Medicine 31.0%, together 31% of the corpus, against Immunology at 74.2% -- and systematic: "Molecular Biology" behaves as an attractor, absorbing papers belonging to Biochemistry, Cancer Research, Genetics, Biotechnology and Cell Biology.

OpenAlex has documented the upstream cause. In 2026 it merged 27,728 duplicate journal source records affecting over 6 million works, shrinking its source catalogue from 282,924 to 255,434. Its topic model uses venue as an input signal, so works whose journal was fragmented across records were classified against a partial view of that venue.

Does relabelling actually help?

Scoring a model trained on labels X against labels X is circular. The comparison below uses held-out DataCite dataset records, whose labels come from a third annotation produced by neither training pipeline. Both models had identical exposure to DataCite during training (70% mixed in, same split, same seed), and both are scored on the held-out 30%.

Better on every metric, and the macro gain (+9.3) is larger than the micro gain (+3.1), meaning most of the improvement is in the long tail rather than in the already-easy common subfields. Eleven fewer subfields have zero recall.

It also generalises rather than memorising its own labels: scored against OpenAlex's held-out labels it reaches field accuracy 0.9377, higher than the OpenAlex-trained model's own 0.8694. It beats that model at OpenAlex's field assignments while disagreeing about subfields.

On its own held-out labels

subfield top-1 0.6426, top-3 0.8697, field top-1 0.8758, field top-2 0.9675, domain 0.9474, over 247 subfields.

Limitations, stated rather than buried

Half the adjudicated labels were unusable and are not in this model. The labelling prompt was given each paper's OpenAlex field and only that field's subfields. Where OpenAlex's field is wrong -- 48.5% of rows, by its own disagreement with a DeepSeek field re-annotation -- the adjudicator was picking the best option from the wrong list. Those rows are dropped. Training used the 88,887 papers whose field two independent sources agree on. Relabelling the remainder conditioned on a corrected field would roughly double the training set and is the obvious next improvement.

Training was not sample-balanced. Class imbalance is handled by class-balanced ("effective number") loss weights, not by equalising rows per subfield. The training distribution runs from 1 row to 2,541 with a median of 176, and 57 subfields have fewer than 50 rows. That is the main reason macro accuracy is 0.683 rather than approaching micro: the tail is thinly supported. Per-class capping was tested on the sibling model and rejected -- it raises macro but costs more micro than it gains.

subfield_name is not a unique key in the OpenAlex taxonomy. Seven names exist in two different fields each: Biochemistry, Genetics, Microbiology, Neurology, Pharmacology, Physiology, Archeology. Resolving a label by name alone silently mislabels roughly 7.7% of rows and can drop whole classes from a model. Always key on subfield_id, or on (field_name, subfield_name).

Subfield accuracy is agreement with an LLM adjudication, not human ground truth. No human labelling was performed. The direction is supported by the attractor pattern, by the adjudicator disagreeing more where it is less confident (48.1% agreement at confidence >= 0.9 against 28.4% below 0.7), and by OpenAlex's documented upstream defect -- but it is not proof.

DataCite is a fair arbiter, not an independent one. Its labels come from a third annotation, and the split is held out from both models, but both models trained on 70% of that corpus, so it measures out-of-sample generalisation rather than performance on a wholly unseen annotation style.

Usage

from subfield_classifier import SubfieldClassifier

clf = SubfieldClassifier()
clf.classify(title="...", abstract="...")

Input is title + the first 1,200 characters of the abstract; feeding less degrades it. Field probability is the sum of its subfields' probabilities and domain is a lookup from the top field, so the returned hierarchy is legal by construction. Temperature applies only to the returned uncertainty signal, never to the emitted labels.