augmented-residuals-culturax24-downstream

CulturaX-24 XNLI and PAWS-X full-finetuned checkpoints

Six sequence-classification models obtained by full fine-tuning the 115,200-step CulturaX-24 (40 GB, SentencePiece 120k) plain residual, four-stream mHC-lite, and full Attention Residuals language-model checkpoints.

Training used English examples for five epochs (effective batch size 32, learning rate 2e-5, seed 42). Evaluation is multilingual over all task test languages. Values below are accuracies in percent; test macro is the unweighted mean over languages.

Files

Each <architecture>/<task>/ directory contains:

• model.pt: a PyTorch dictionary with model_state_dict, summary, and label_names. The state dict covers the full custom backbone plus the sequence-classification head.
• summary.json: training configuration, validation history, and per-language evaluation metrics.

The architecture directory names are plain, mhc-4stream, and attnres-full; task names are xnli and pawsx.

These are project-native PyTorch checkpoints, not Transformers save_pretrained directories. Load the corresponding 115,200-step backbone with the multi-mhc code, construct eval.models.SequenceClassifier, and restore model_state_dict. The shared SentencePiece tokenizer is the one distributed with the CulturaX-24 pretraining setup.

The complete evaluation and training metadata needed to distinguish these runs from linear probes is included in every summary.json.