presence-counter-400m

presence-counter-400m

Reads a passage of fiction or roleplay and answers one question: how many people are physically in this scene right now?

The answer is a single number from 0 to 6, where 6 means "six or more". No roster, no spans, no explanation — just the count of bodies standing in the room at the end of the passage.

Ashley poured the tea while Marcus watched.          -> 2
Ashley poured the tea and thought about Marcus.      -> 1
Ashley poured the tea. "Marcus is late," she said.   -> 1
The kitchen was empty. Marcus had left his coat.     -> 0

Why this is harder than it looks

Almost all of the difficulty is in what does not count. A name in a passage is weak evidence that its owner is in the room. People get talked about, remembered, phoned, photographed, written to, dreamt about, and heard through walls — and a model that simply tallies capitalised words gets all of those wrong while still scoring well on ordinary prose.

So the model is graded on an eval built specifically to catch that shortcut: passages that differ by one sentence, where a character is bodily present in one version and merely mentioned in the other, with the same names in both. A pure name-counter gives the same answer to both halves of every pair. This model gives the same answer to 5.6% of them.

What counts as present

Counted: anyone whose body is in the depicted space at the end of the passage — acting, speaking, listening, asleep, or silently watching from a corner. Unnamed people count if they are individuated ("two guards", "the woman with the clipboard"). The "I" of first-person narration counts. The "you" of second-person narration counts. Someone who walks in halfway through and stays counts.

Not counted: anyone who is only talked about, thought about, remembered, imagined, shown in a photograph or a reflection, heard over a phone or radio, audible through a wall from the next room, present in a flashback, or gone before the last sentence. Animals, corpses, statues, mannequins and machines are not people. A name and a role can be the same person — "Sergeant Vance" and "Vance" is one body.

A few cases are genuinely arguable, so the model follows a fixed policy rather than guessing: a voice on a phone is absent, someone in the next room is absent, someone who leaves before the end is absent, an underspecified plural takes the smallest count the text supports, and a crowd too large to enumerate is 6.

Using it

transformers

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

name = "chartreuse-verte/presence-counter-400m"
# truncation_side="left" matters: when input overflows, keep the END of the scene,
# because the question is who is here *now*.
tok = AutoTokenizer.from_pretrained(name, truncation_side="left")
model = AutoModelForSequenceClassification.from_pretrained(name).eval()

def count_people(text: str) -> int:
    enc = tok(text, truncation=True, max_length=1024, return_tensors="pt")
    with torch.no_grad():
        return int(model(**enc).logits.argmax(-1))

count_people("Ashley poured the tea and thought about Marcus.")   # 1

GGUF (CPU)

gguf/presence-400m-q8_0.gguf is a 403 MiB q8_0 build, converted with llama.cpp's convert_hf_to_gguf.py and quantised with llama-quantize. It carries the modern-bert architecture, 174 tensors, and the classification head is in the file (cls.weight, cls.norm.weight, cls.output.weight, cls.output.bias), so a llama.cpp build with ModernBERT classifier support can score with it directly. The numbers reported below were produced with the PyTorch weights, and the ONNX export is the one that has been checked against them numerically.

ONNX

onnx/model.onnx is fp32 and agrees with the PyTorch weights to a maximum logit difference of 1.0e-05, with identical argmax on every case checked.

Input length

The window is 1024 tokens, which covers a multi-paragraph scene. Longer input should be clipped from the end, keeping the most recent text: the label is a property of the final moment of the passage, so keeping the beginning answers a question about an earlier point in the scene.

How it was built

The training corpus is 17,347 synthetic scenes, written label-first. Natural prose is overwhelmingly "two people talking" — the scenes that teach the actual distinction (nobody present but four people discussed; five bodies in one room; a scene whose only named character is on the phone) barely occur, so they were commissioned instead. A target count is chosen first, a scene is written to order, and that scene is then labelled from scratch by models that never saw the target. Every cell of the 7 × 4 grid of (people present × people mentioned-but-absent) holds at least 149 scenes.

The scenes come from seven writer models across four vendors — deepseek-v4-pro (4,358 scenes), MiniMax-M3 (2,524), deepseek-v4-flash (1,840), gemma-4-26b (1,541), gpt-5.4-mini (1,252), Inkling-Small (1,220) and muse-spark-1.2 (755) — plus 3,857 from the earliest round, which do not record their writer. One writer means one set of habits: the same paragraph rhythm, the same names, the same way of walking a character into a room.

Each brief is assembled from independently sampled axes, so none of them can become a shortcut to the count. Across the scenes that record theirs, the corpus draws on 164 settings, 141 archetypes, 55 genres, 58 name cultures, 40 narrative registers and 25 eras, plus three axes that exist to make the count hard to read: a complication (an unnamed silent bystander, a headcount spoken aloud that is wrong, unattributed dialogue), a staging note (where the bodies are: in a doorway, up a ladder, behind a counter) and a noise note (capitalised words that are not people: a named boat, a storm, a dog, a brand). 519 scenes contain no proper names at all — the only rows where counting capitalised words cannot work even in principle. 2,540 were written past the window on purpose and clipped from the end, so the model sees passages that begin mid-argument as well as scene openings.

No label is hand-written, and no label comes from a single model. Each scene was read by two different LLMs which returned a roster — who is present, who is absent — rather than a number, because naming people is something language models do well and counting is not. Agreement between the two is the label; a one-apart disagreement resolves to the lower count; anything further apart is thrown away. No scene is ever labelled by the model that wrote it.

Base model: jhu-clsp/ettin-encoder-400m. Fine-tuned for 6 epochs at lr 3e-5, effective batch 32, with a little target mass on the neighbouring counts so the ordered classes are treated as ordered; the released weights are the epoch with the best held-out macro-F1. 15,648 scenes were used for training and 1,699 held out, split so that scenes commissioned from one request cannot straddle the split.

How well it works

Two eval sets, reported separately because they answer different questions.

Constructed probes (176 items, 90 contrastive pairs) — does it apply the rule, or has it learned that a name means a body? The gold answers follow from how the items were built, not from any model's opinion.

Accuracy holds steady as distractors pile up — 91.7% with no absent people mentioned, 96.3% with three or more — which is the confounder failing to bite. Per-trap: photographs, disembodied voices, letter writers, flashbacks, thought-about people and numeral distractors are all at 100%; the weakest are people audible through a wall (86.7%), dream figures (86.7%) and characters who leave midway (87.5%).

Real prose (495 scene windows from a roleplay chat database, labelled once by the strongest available model) — does it work on the distribution it ships into?

Real conversation is lopsided: 73% of those windows contain exactly two people, so exact accuracy is the wrong headline and macro-F1 and MAE are the honest ones. Per class, F1 is 0.88 at two people and 0.76 at one, dropping to 0.25–0.52 for the rare counts, each of which has 6–21 rows behind it.

On real prose, unlike on the constructed set, accuracy does slide as more absent people are mentioned: 82.1% when nobody absent is referred to (374 rows), 68.8% with one (80), 62.1% with two (29), 58.3% with three or more (12). Free prose mentions absent people in messier ways than a template can, and those rows are where the model is most likely to be wrong.

Nine hand-written cases covering every rubric ruling — the tea/Marcus triple, a voice on the radio, a photograph of four cadets, an empty kitchen, unattributed dialogue, a crowd, and a group whose fourth member is away in the city: 9/9.

Limitations

• Crowds are the weak class. The constructed eval has four items at "six or more" and the model misses all four, answering 1, 2, 5 and 5. On real prose it gets 31% of the 13 crowd rows exactly right. If you need reliable crowd detection, treat anything ≥ 4 as "a lot" rather than trusting the exact number.
• The arguable rulings are policy, not physics. If your application wants a voice on the intercom or a person in the next room to count, this model will disagree with you consistently rather than randomly.
• English only, and the training scenes are prose fiction and roleplay. Screen directions, transcripts and dialogue-only formats appear in the corpus; legal, technical and encyclopedic text does not.
• No human labelled any of it. The training labels come from two models agreeing and the real-prose eval labels from one strong model. The constructed eval is the exception — its answers come from how each item was assembled. The pipeline is self-consistent, which is not the same thing as correct.
• Under-count exceeds over-count on the constructed eval (5.7% vs 2.3%): when it is wrong, it is more often missing a quiet body than counting a mentioned one.
• Rare-count numbers on real prose rest on very few rows. Classes 0, 3, 4, 5 and 6 have 12, 21, 10, 6 and 13 rows respectively.

Files