kompass-followup-model

kompass-followup-model

Cross-encoder for chapter boundaries inside one news story: given two articles already known to belong to the same story, does the later one report a new development?

p(NEW_STEP) = softmax(logits)[1]     labels: SAME_DEVELOPMENT, NEW_STEP

text_a must be the earlier article. NEW_STEP is directional and the model was trained in that order; swapping the arguments changes the answer.

Base: deepset/gbert-base, fine-tuned on development_relation_training_v2.

It is a veto, not a detector

Used alone this model is worse than doing nothing: on a held-out set of three news stories it scores F1 0.396 against 0.450 for the null model that puts every article of a story in one chapter.

It earns its place only in conjunction with a time gap:

a new chapter iff  gap since the previous article >= 4h
             AND   p(NEW_STEP) >= 0.05

one chapter per story (null)   0.450
model alone                    0.396
time gap alone                 0.703   (collapses to 0.343 on one story)
BOTH                           0.778

Its real job is stopping the clock from opening a chapter when two articles hours apart are the same wire copy. That is a much smaller job than "detect a development", and it is the one it can do.

fp32 only, on purpose

onnx/model_quantized.onnx is not published. INT8 flips 14 of 250 decisions at the 0.05 operating point, which sits in the crowded low end of the score distribution. transformers.js loads the quantized file by default, so omitting it makes a caller that forgets quantized: false fail loudly instead of quietly serving a different model.

const model = await AutoModelForSequenceClassification.from_pretrained(
  'GJOE/kompass-followup-model', { quantized: false });

Limits

Measured on 6 German news stories (~250 articles), labelled by one annotator from headlines, 3 tuned on and 3 held out. Worst held-out story: 0.600. It under-splits when coverage is continuous.