Downloads · 30 days
2
13% of all-time downloads
token-averaging-avg_250m_k2
FAIRC/token-averaging-avg_250m_k2
token-averaging-avg_250m_k2 is a machine learning model from FAIRC. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Checkpoint dump from the token averaging research project.
All-time downloads
16
Public
Repo size
20.6 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt20.6 GB · 100%
At a glance
- Access
- Public
- Created
- Aug 2, 2026
- Updated
- Aug 2, 2026
- SHA
- 84d89a85
Model card
From the Hugging Face model README
FAIRC/token-averaging-avg_250m_k2
Checkpoint dump from the token averaging research project.
- run name:
avg_250m_k2 - results tree:
results
Contents
Loss logs
loss_log.csv
Checkpoints
checkpoints/final.ptcheckpoints/step_00050000.ptcheckpoints/step_00100000.ptcheckpoints/step_00150000.ptcheckpoints/step_00200000.ptcheckpoints/step_00250000.ptcheckpoints/step_00300000.pt
Loading a checkpoint
import torch
from huggingface_hub import hf_hub_download
path = hf_hub_download('FAIRC/token-averaging-avg_250m_k2', 'checkpoints/final.pt')
state = torch.load(path, map_location='cpu', weights_only=False)
model.load_state_dict(state['model']) # your OLMAveraged / OLMTransformerBody
print(state['step'], state['tokens_seen'], state['cumulative_flops'])
These are not Hugging Face transformers weights. Rebuild the
architecture from config.json → model_config (or from
experiments/chinchilla/model_configs.py in the source repo) and load
the raw state_dict.
Architecture
{
"d_model": 1024,
"n_heads": 16,
"n_layers": 16,
"context_len": 1024,
"averaging_k": 2,
"tie_embeddings": true,
"lr": 0.00014,
"warmup_steps": 2000,
"target_tokens": 10000000000,
"n_params_approx": 252789760
}
- Created
- Aug 2, 2026
- Updated
- Aug 2, 2026