TR-HASH-MOE-500M-HF

TR-HASH-MOE-500M-HF — semantic-cell LoRA experiment

Failed promotion gate — research artifact only. This checkpoint is published for reproducibility and diagnosis. It is not a safe or reliable assistant and must not be used for consequential decisions.

Experimental merged checkpoint from step 1,052, trained for one full-shard epoch from the clean 500M/20B pretrained base. The LoRA weights are already merged; no separate adapter is required.

Training state

The bundle contains merged BF16 weights, the 32K tokenizer, the standalone chat template, and the fixed layer-specific top-2 route tables.

Inference

pip install git+https://github.com/Complexity-ML/vllm-i64.git@main

vllm-i64 serve tr-hash-moe-500m \
  --checkpoint AETHORIA-AI/TR-HASH-MOE-500M-HF \
  --dtype bfloat16 \
  --port 7860

Limitations

This checkpoint failed the behavioral promotion gate. Full PIQA normalized accuracy was retained (0.6953 base -> 0.6964 candidate), but the fixed nine-prompt assistant panel failed all nine behaviors. Observed failures include incorrect arithmetic, prompt copying, fabricated identity details, ignored format constraints, repetitive text, incomplete factual definitions, and an unsafe password-sharing answer.

The result indicates an instruction-data coverage limitation rather than a general PIQA capability regression. Loss weighting cannot create semantic or stylistic coverage that is absent from the shard. Do not interpret the lower matched-eval loss as evidence of assistant quality.

No safety, correctness, or production-readiness claim is made. Users should retain an independently validated base or checkpoint for any real application.

Resources

• Training framework
• Custom runtime
• Post-training dataset