retinalai-narrator-135m

RetinalAI compact narrator (135M, vocabulary-pruned)

Writes the free-text clinical narrative for a retinal screening result. It is a research artifact, not a medical device — see Limitations.

Distilled (sequence-level KD) from a self-hosted Qwen/Qwen3-8B-AWQ teacher onto SmolLM2-135M-Instruct, then vocabulary-pruned for footprint.

What is different about it

• Prose only. It emits the narrative as plain text. The structured triage decision is served separately by a 3 KB linear head, so this model has no JSON contract to break. An earlier JSON-emitting version of the same base model failed to parse on 100% of cases once 4-bit quantized; prose-only generation is 1.000 at every precision.
• Pruned vocabulary, 49,152 -> 920 tokens. Embedding rows are the original tokenizer ids listed in keep_ids.json, in order. The tokenizer is unmodified; callers map ids old->new before the model and new->old before decoding. Tokens outside the kept set are re-encoded byte-wise rather than dropped. The vocabulary is seeded from all 45 disease names the upstream classifier can emit, and all 45 are verified to survive the round trip.

Use src/narrator/compact.py from the OptiscanAI repository — the id remap is required, so AutoModelForCausalLM alone will produce garbage.

Evaluation

24 held-out cases, scored against the teacher's own narrative for the same case:

bf16 is the only precision recommended. 4-bit reaches 54 MB but drops findings the teacher reported in 42% of cases while remaining fluent, which makes the omissions hard to notice.

"Acuity divergence" counts narratives using urgency language the teacher did not use for that same case. It is a broad screening signal, not a hallucination count.

Limitations

• Not a medical device. Not for clinical use. No regulatory clearance, no clinical validation, no prospective study.
• Evaluated on 24 held-out cases from a single RFMiD-derived sample. That is enough to distinguish "works" from "broken" and little else.
• No clinician review of generated text has been performed.
• Trained on 56 examples. It reproduces the teacher's register on cases that look like its training distribution and has not been probed outside it.
• No EMERGENCY case appeared anywhere in the source data, so emergency phrasing is untested. Escalation is handled deterministically upstream, not here.
• Callers must append their own AI-disclosure statement; the teacher traces did not contain one, so the model does not produce it.