Experiment-1-A-xIELU-CPT

Experiment-1-A xIELU CPT

Continued pretraining of the completed Experiment-1-A checkpoint with pure trainable xIELU from CPT step 0, on the next unseen FineWeb-Edu shards. The repository contains custom PyTorch source and raw safetensors weights; it is not a drop-in Transformers model.

Status

• Status: complete
• Base processed tokens: 2,000,158,720
• CPT processed tokens: 2,000,158,720
• CPT unique tokens: 2,000,158,720
• Total lineage processed tokens: 4,000,317,440
• Known unique usable exposure: 2,999,910,400
• Next zero-based training step: 3815
• Run ID: 383b8514-3ae7-4174-b21d-586ef0bf88c9
• GPU: NVIDIA A100-SXM4-40GB
• Training dtype: BF16
• Target tokens: 2,000,158,720
• Tokens per optimizer step: 524,288
• Base lineage source: efe-T/Experiment-1-A/checkpoints/final/model.safetensors
• Base SHA-256: 357a6a3f1abfeaf433a5425581346aac018fe338339e20ebb50e0b1fc312e9fa
• Data: pinned shards 000011 through 000031, monotonic and non-wrapping
• Activation: pure trainable xIELU from CPT step 0

Repository layout

• model/model.safetensors: final standalone model weights
• checkpoints/250M, 500M, 750M, final: retained full checkpoints
• checkpoints/latest/manifest.json: pointer to the newest valid resumable checkpoint
• training/: exact source and resolved training configuration
• runtime/: environment metadata
• metrics/training_metrics.jsonl: per-step training metrics

FineWeb-Edu dataset shards are not included.