Downloads · 30 days
7
8% of all-time downloads
cminst/realistic-bpe5-union-longest-specialist-500m-10b
realistic-bpe5-union-longest-specialist-500m-10b is a machine learning model from cminst. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for pytorch. The card lists the license as other.
Final 500M Union-Longest specialist checkpoint trained on the single longest-match tokenizer route over the RealisticBPE5 string union.
Downloads · 30 days
7
8% of all-time downloads
All-time downloads
92
Public
Repo size
29.4 GB
Likes
0
Public
Click a slice to open those files.
.pt29.4 GB · 100%
From the Hugging Face model README
Final 500M Union-Longest specialist checkpoint trained on the single longest-match tokenizer route over the RealisticBPE5 string union.
HF repo: cminst/realistic-bpe5-union-longest-specialist-500m-10b.
Source run directory name: R5FW10B-SpecNA-realistic_bpe5_union_longest-500M-10B-local.
Data bundle: cminst/realistic-bpe5-union-longest-10b.
model.pt: final model weightsconfig.json: training/model configuration saved by train_v3.pyresults.json: final evaluation metrics and logsstep_*/: intermediate model-only checkpoints (model.pt, config.json, results.json, and string_atlas.json when present)tokenizers/*.json: tokenizer JSON filesrealistic_bpe5_union_longestrealistic_bpe5_union_longest: BPB 0.8940 at step 9542{
"anchor": "realistic_bpe5_union_longest",
"anchor_weight": 1.0,
"completed_steps": 9542,
"ctx_len": 2048,
"d_model": 1536,
"data_format": "packed_tokens",
"input_mode": "full",
"lr_decay_steps": 9542,
"model_size": "500M",
"n_heads": 24,
"n_layers": 18,
"output_mode": "full",
"run_name": "R5FW10B-SpecNA-realistic_bpe5_union_longest-500M-10B-local",
"tokens_processed": 10000626688,
"vocab_sizes": {
"realistic_bpe5_union_longest": 50184
}
}