Downloads · 30 days
9
13% of all-time downloads
cminst/realistic-bpe5-union-longest-specialist-1b-20b
realistic-bpe5-union-longest-specialist-1b-20b is a machine learning model from cminst. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for pytorch. The card lists the license as other.
Final 1B Union-Longest specialist checkpoint trained on the single longest-match tokenizer route over the RealisticBPE5 string union.
Downloads · 30 days
9
13% of all-time downloads
All-time downloads
68
Public
Repo size
102 GB
Likes
0
Public
Click a slice to open those files.
.pt102 GB · 100%
From the Hugging Face model README
Final 1B Union-Longest specialist checkpoint trained on the single longest-match tokenizer route over the RealisticBPE5 string union.
HF repo: cminst/realistic-bpe5-union-longest-specialist-1b-20b.
Source run directory name: R5FW20B-spec-realistic_bpe5_union_longest-1B-20B-tillicum.
Data bundle: cminst/realistic-bpe5-union-longest-20b.
model.pt: final model weightsconfig.json: training/model configuration saved by train_v3.pyresults.json: final evaluation metrics and logsstep_*/: intermediate model-only checkpoints (model.pt, config.json, results.json, and string_atlas.json when present)tokenizers/*.json: tokenizer JSON filesrealistic_bpe5_union_longestrealistic_bpe5_union_longest: BPB 0.8504 at step 19083{
"anchor": "realistic_bpe5_union_longest",
"anchor_weight": 1.0,
"completed_steps": 19083,
"ctx_len": 2048,
"d_model": 2048,
"data_format": "packed_tokens",
"input_mode": "full",
"lr_decay_steps": 19083,
"model_size": "1B",
"n_heads": 32,
"n_layers": 20,
"output_mode": "full",
"run_name": "R5FW20B-spec-realistic_bpe5_union_longest-1B-20B-tillicum",
"tokens_processed": 20000205312,
"vocab_sizes": {
"realistic_bpe5_union_longest": 50184
}
}