trace-checkpoints

TRACE — contextual turn encoders (REPRO campaign)

Checkpoints for TRACE, a contextual turn encoder that learns the update rule applied to frozen turn embeddings in task-oriented dialogue.

A frozen base encoder $f_1$ maps each utterance to a static vector $e_t$; TRACE ($f_2$) is a Transformer whose tokens are turns, and it maps the sequence $(e_1,\dots,e_T)$ to contextual representations $h_t$. Training is self-supervised and requires no functional annotation.

All checkpoints come from the REPRO campaign: a pre-specified protocol in which dialogues are partitioned by near-duplicate cluster, probes are fit on a held-out development split, and test labels never inform training, checkpoint or hyperparameter selection. Every model was retrained from scratch under that partition.

Which one to use

The AR mode is the one to use in practice: it only attends to the dialogue so far. Bidirectional checkpoints see the whole window and are included for the representation-vs-anticipation analysis.

Naming

trace-repro-<recipe>-<mode>-s<seed>[-xlc][-scratch]
             │        │      │      │      └── trained from scratch on the 28M curriculum
             │        │      │      └───────── continued pretraining on ~28M turns
             │        │      └──────────────── training seed: 42, 7, 123
             │        └─────────────────────── ar | bidi
             └──────────────────────────────── lite (6 layers) | deep (12) | gru (recurrent control)

Base-ablation checkpoints carry the base in the name instead: mpnet, todbert.

Correspondence with the paper

Untrained controls are not published: they are randomly initialized copies of the same architectures, reproducible from the configs with seeds 0–4.

Contents of each folder

<checkpoint>/
├── best/               # selected by validation loss — use this one
│   ├── config.json
│   └── model.safetensors
├── config.json         # final epoch
├── model.safetensors
└── trainlog.jsonl      # per-epoch train/val loss

Usage

from contextual_turn_embeddings import ContextualTurnModelV2, encode_dialogues

model = ContextualTurnModelV2.from_pretrained("best", device="cpu").eval()
# embeddings: (n_turns, 768) from the frozen base, in dialogue order
H, meta = encode_dialogues(model, frames, embeddings=E, device="cpu")

The package lives at packages/contextual-turn-embeddings in the project monorepo, together with the partition, the training and evaluation scripts, the seeds and the commands that regenerate every table.

Base turn embeddings

The precomputed $e_t$ vectors used as input are published separately as datasets: jumafernandez/d2f-turn-embeddings-*.

Citation

Fernández, J. M., Errecalde, M., & Burdisso, S. TRACE: Learning the Update for Contextual Turn Representations in Task-Oriented Dialogue. (under review)