TR-HASH-MoE-200M-130B-Checkpoints

TR-HASH MoE 200M — 70B Unique / 130B Replay Checkpoints

Status: training complete. This is the raw checkpoint backup repository, not the finished model release. It exists as a safety net and a record of the training trajectory — token-pack folders include model weights and resumable optimizer/scheduler state.

• Checkpoints are uploaded automatically at token-pack boundaries and on clean/interrupted shutdown by scripts/sync_checkpoints_to_hf.py.
• Folder names follow {tag}_{step} (token_pack_NNN_STEP, final_STEP, interrupted_STEP).
• The PIQA sweep below is exploratory, not a full validated evaluation suite. Do not treat an individual checkpoint as a finished model release.
• The architecture config is tracked at model_config.yaml in this repo (not embedded in checkpoint.pt).

Exploratory zero-shot checks (not a full evaluation)

Informal checks across the final token-pack checkpoints, via scripts/convert_to_mlx.py + scripts/eval_mlx_zero_shot.py, zero-shot causal-log-likelihood scoring, no chat template. Every row uses the same tokenizer, MLX FP16 inference path, PIQA validation split, and 1,838 examples:

token_pack_035_144636 has the highest acc_norm in this sweep. The trajectory is not monotonic, and PIQA alone must not be used as a complete model-quality or checkpoint-selection criterion.

The finished base model is published separately as AETHORIA-AI/TR-HASH-MoE-200M-130B.

See Complexity Framework for the training code.