notes-slots

notes-slots

Compact token-classification model that extracts scheduling/task slots from short English notes — participants, datetimes, priorities, and recurrences — and runs fully client-side via Transformers.js.

The shipped artifact is an INT8-quantized ONNX bundle (~7.6 MB) intended for in-browser WASM inference, not a PyTorch checkpoint.

Model details

Labels (9, BIO)

O, B-PARTICIPANT, I-PARTICIPANT, B-PRIORITY, I-PRIORITY, B-DATETIME, I-DATETIME, B-RECURRENCE, I-RECURRENCE

A bundled transitions.json carries a BIO-constraint transition matrix (0 for allowed transitions, a large negative penalty for invalid I- continuations) for constrained Viterbi decoding on top of the raw token logits. The bundled JS loader applies it automatically. The matrix is deliberately constraint-only: empirical bigram priors double-count label frequency already present in the emissions and were measured to cost ~7 F1 points of recall on the quantized model.

Intended use

• In scope: extracting participants / datetimes / priorities / recurrence cues from short, informal English notes and reminders (calendar, to-do, email intent style text).
• Out of scope: long documents, languages other than English, normalization of extracted spans into structured datetimes (use a downstream parser such as chrono-node for that), and any high-stakes decisioning.

Usage (Transformers.js)

import { pipeline } from "@huggingface/transformers";

const tagger = await pipeline("token-classification", "jottypro/notes-slots", {
  dtype: "q8",
});

const out = await tagger("call Sarah next Friday at 5pm, high priority, every week");
console.log(out);

The ONNX weights live at onnx/model_quantized.onnx, which is the layout Transformers.js expects when loading from the Hub.

Training

• Data: AmazonScience/MASSIVE en-US (config en-US, revision d2362678…), filtered to the calendar / datetime / email / lists scenarios with MASSIVE slots remapped onto the local 4-slot schema (e.g. person/relation/email_address → PARTICIPANT, date/time/time_zone → DATETIME, general_frequency → RECURRENCE), combined with synthetic productivity and realistic note generators.
• Augmentation: light, training-split only (AUGMENT_FACTOR=2) — random filler-word prefix, trailing punctuation, occasional O-token dropout; deduplicated against originals.
• Hyperparameters: 10 epochs with early stopping (patience 2, restore best by F1), batch size 64, learning rate 5e-5, cosine schedule, warmup ratio 0.1, weight decay 0.01, label smoothing 0.05, max sequence length 128, seed 42.
• Distillation: the shipped student is knowledge-distilled from a bert-base-uncased teacher fine-tuned on the same data (teacher eval F1 ~0.90); loss = 0.5·CE(gold) + 0.5·T²·KL(teacher‖student), T=2.
• Vocab pruning: wordpiece vocab pruned 30522 → 9567 (pieces observed in domain data, fixtures, regression texts, plus a wikitext-103 frequency sweep; all single-char and digit pieces retained). Pruned tokenizer emits token-identical output on the full test set, so metrics are unchanged.
• Quantization: dynamic, per-channel QInt8 with reduce_range, applied to MatMul and Gather ops via ONNX Runtime. Static QDQ quantization was evaluated (MinMax/Entropy/Percentile calibration) and consistently lost 8-12 F1 points on this model size; dynamic + reduce_range won the matrix. Weight-only QAT (straight-through fake-quant fine-tune) was also evaluated and rejected: it landed below the post-training baseline (0.823 vs 0.828) because it cannot model ORT's runtime dynamic activation quantization.

Evaluation

Token-level metrics (seqeval) on the held-out test split (n ≈ 559). fp32 is argmax-decoded; q8 uses the constrained-Viterbi decode that ships in the JS loader. The q8 column reflects the artifact actually shipped in this repo.

Limitations and bias

• Quantization cost: with dynamic per-channel INT8 (reduce_range) and constrained-Viterbi decoding the gap to fp32 is ~2 F1 points (0.850 → 0.828). Decoding with plain argmax instead of the bundled transitions costs a further ~2 points; empirical-prior Viterbi costs ~7. PRIORITY is the most quantization-sensitive entity (0.75 → 0.67).
• Domain: trained on short calendar/task-style English notes plus synthetic data; expect degradation on long-form text, other domains, or other languages.
• Synthetic data: part of the training distribution is generated, so phrasing diversity and demographic coverage of names/relations is limited and may carry generator biases.
• No span normalization: the model tags spans only; converting a DATETIME span to an actual timestamp is a downstream concern.

License and attribution

Released under CC BY 4.0, consistent with the MASSIVE training data (CC BY 4.0). The base model microsoft/xtremedistil-l6-h256-uncased is MIT.

Part of the training data is derived from the MASSIVE dataset; CC BY 4.0 requires attribution to that source:

@misc{fitzgerald2022massive,
  title  = {MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languages},
  author = {FitzGerald, Jack and others},
  year   = {2022},
  eprint = {2204.08582},
  archivePrefix = {arXiv}
}