unquote-small

Unquote-small

Strip the quoted history off an email without deleting the part you wrote.

Quoted replies, signatures and legal boilerplate are most of a mail thread and none of its meaning. Removing them before you pay a model to read the thread is easy money — right up until the cleaner eats a paragraph the sender actually wrote, and you have no way to notice.

unquote-small is a 31M-parameter CPU model that tags every line of an email as one of eight zones. It is built around one asymmetry: leaving quoted text in costs you tokens; taking body text out costs you the message. Every decision is biased toward keeping.

Results

Measured on 1,200 held-out messages (synthetic + real public mailing-list mail). Splits are disjoint by thread, so no reply shares a split with the parent it quotes.

body loss is the fraction of true BODY lines a system deletes — the number that decides whether a cleaner is safe to run unattended.

By language (synthetic split)

Scored on the synthetic split because its labels are correct by construction. Non-English real mail in the evaluation corpus has pattern-derived gold for the trailer zones, which would be circular to score a rule-based baseline against.

Rule-based extractors carry English rule tables; this model has no such centre of gravity. English-vs-rest macro-F1 gap: talon +0.049, Unquote −0.059.

Use

from unquote import ZoneTagger
tagger = ZoneTagger.from_pretrained("NagaYu/unquote-small")
print(tagger.clean(raw_email))            # body only

pip install "unquote[export] @ git+https://github.com/NagaYu/unquote"
unquote clean mail.txt --keep body,attribution

Files in this repo: model.pt (torch), model.onnx (int8, 6.4 ms/msg), model.fp32.onnx, tokenizer.json, config.json, gate.json (the fitted safety thresholds), sweep.json (the full calibration curve).

What it promises

It will not delete your body text, bought in four places you can inspect:

That last row is the honest one. A confidence gate cannot stop a confidently wrong prediction. On deliberately damaged mail at threshold 0.999 the measured body loss was mean 0.0005 / worst 0.0625, affecting 1 message in 120. The claim is "fitted to zero body loss and measured on held-out data", not an impossibility proof.

It does not promise a perfect split. Leftover quoted lines are an accepted outcome of that trade. If you need every quoted byte gone and can tolerate losing prose, this is the wrong tool.

Choosing an operating point

The gate ships fitted for zero body loss. That is one point on a measured curve:

The curve is steep — accepting roughly one lost line in 2,200 nearly doubles the savings. Move along it with --threshold, or refit with scripts/train.py --target-body-loss 0.002.

Cost of the strict end: at zero body loss the gate rescues signatures the model was 89% sure about, so SIGNATURE F1 falls from 0.897 ungated to 0.429 gated. That is the trade working as designed, but it leaves more signature text in place than the model is capable of removing.

The eight zones

BODY (never removable), QUOTE, ATTRIBUTION, SIGNATURE, DISCLAIMER, MOBILE_FOOTER, AUTOREPLY, FORWARD_HEADER.

Two conventions: everything inside a quoted message collapses to QUOTE (including its signature — a quoted signature is not this author's), and the introducer line keeps its own zone at every nesting depth.

Architecture

tokens of one line -> embedding -> shift-mix -> masked mean+max pool
concat 88 structural features (quote depth, script, wrap width, distance from end)
-> transformer over the LINE sequence        <- where the evidence actually is
-> per-line emissions + learned transitions
-> Viterbi over (zone, seen_trailer)         <- "no body after a signature", enforced

8 document layers, d_model 512, vocab 21,079, 31.1M parameters. Position is encoded from both ends: trailer zones are anchored to the end of a message, so the encoder gets distance-from-end alongside distance-from-start.

Decoding runs Viterbi over the augmented state (zone, seen_trailer), which latches once a signature/disclaimer/mobile-footer line is emitted; BODY is -inf in the latched half. No body line can follow a signature no matter what the network scores — including via the SIGNATURE -> QUOTE -> BODY path that a pairwise transition ban cannot block.

Training data

Public mailing-list archives only — Apache (lists.apache.org), Python (mail.python.org), GNU (lists.gnu.org) and the localized Debian lists in 15 languages — plus synthetic threads from unquote.compose whose labels are correct by construction.

Excluded on purpose: Enron and every corpus like it. Freely downloadable, and still a collection of private messages whose authors never consented to publication. Public availability is not consent. The dataset builder enforces a deny-list.

Privacy. Every real message is scrubbed before training: addresses become RFC 2606 reserved domains, phone numbers move to the +1-555-01xx fictional range, and every participant's name is replaced — not just the sender's, since attribution lines and quoted signatures name third parties. Archive-obfuscated addresses (a <at> b.com) are handled too. Replacements are structure-preserving, so the tagging task is unchanged.

Limitations

• Throughput is tens to hundreds of messages/second, not thousands. 17.6 ms/msg fp32, 6.4 ms/msg int8, single process on 6 CPU threads. A 31M transformer over a line sequence does not reach thousands/sec on one core.
• Trained on developer mailing lists. Corporate mail, marketing mail and ticketing-system notifications are out of distribution.
• Real-data gold for SIGNATURE/DISCLAIMER is pattern-derived; those rows are flagged in the benchmark output and are not admissible evidence against a rule-based baseline.
• HTML-only messages are dropped at ingest — without line structure there is nothing to line-tag.
• An earlier checkpoint trained on synthetic data alone scored 0.982 on synthetic and 0.321 on real mail. Withholding real data did not make the evaluation honest, it made the model useless; thread-disjoint splitting is what protects it instead.

Citation and source

Code, benchmark harness, dataset builder and the four guarantee tests: https://github.com/NagaYu/unquote

Licensed Apache-2.0. Archive content remains under the terms of the archives it came from.