Downloads · 30 days
0
atrost/nanochat-d24-matformer-h16-natural-sft-xl
nanochat-d24-matformer-h16-natural-sft-xl is a machine learning model from atrost. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for nanochat.
This is a standalone nanochat-native checkpoint extracted from atrost/nanochat-d24-matformer-h16-natural-sft/d24matformerh16naturalsft at step 000485.
Downloads · 30 days
0
Access
Public
Updated May 7, 2026
Repo size
3.1 GB
Likes
0
Public
Click a slice to open those files.
.pt3.1 GB · 100%
From the Hugging Face model README
This is a standalone nanochat-native checkpoint extracted from
atrost/nanochat-d24-matformer-h16-natural-sft/d24_matformer_h16_natural_sft at step 000485.
matformerXL{"attn_alpha_init_value": 1.0, "dec_alpha_init_value": 1.0, "ffn_alpha_init_value": 1.0, "head_dim": 80, "matformer_ffn_hidden_dims": [5120], "matformer_head_counts": [16], "n_embd": 1280, "n_head": 16, "n_kv_head": 16, "n_layer": 24, "norm_type": "rmsnorm", "sequence_len": 2048, "vocab_size": 32768, "window_pattern": "L"}Load it into the local nanochat checkpoint layout before evaluation:
from nanochat.hf_utils import stage_hf_checkpoint
stage_hf_checkpoint(
repo_id="atrost/nanochat-d24-matformer-h16-natural-sft-xl",
path_in_repo="d24_matformer_h16_natural_sft_xl",
model_tag="d24_matformer_h16_natural_sft_xl",
step=485,
base_dir="/tmp/nanochat-eval",
)
Then run eval with:
NANOCHAT_BASE_DIR=/tmp/nanochat-eval python -m scripts.base_eval --model-tag d24_matformer_h16_natural_sft_xl --step 485