Downloads · 30 days
0
atrost/nanochat-d24-matformer-h16-natural-sft-s
nanochat-d24-matformer-h16-natural-sft-s is a machine learning model from atrost. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for nanochat.
This is a standalone nanochat-native checkpoint extracted from atrost/nanochat-d24-matformer-h16-natural-sft/d24matformerh16naturalsft at step 000485.
Downloads · 30 days
0
Access
Public
Updated May 7, 2026
Repo size
614 MB
Likes
0
Public
Click a slice to open those files.
.pt614 MB · 100%
From the Hugging Face model README
This is a standalone nanochat-native checkpoint extracted from
atrost/nanochat-d24-matformer-h16-natural-sft/d24_matformer_h16_natural_sft at step 000485.
matformerS{"attn_alpha_init_value": 1.0, "dec_alpha_init_value": 1.0, "ffn_alpha_init_value": 1.0, "head_dim": 80, "matformer_ffn_hidden_dims": [640], "matformer_head_counts": [2], "n_embd": 1280, "n_head": 2, "n_kv_head": 2, "n_layer": 24, "norm_type": "rmsnorm", "sequence_len": 2048, "vocab_size": 32768, "window_pattern": "L"}Load it into the local nanochat checkpoint layout before evaluation:
from nanochat.hf_utils import stage_hf_checkpoint
stage_hf_checkpoint(
repo_id="atrost/nanochat-d24-matformer-h16-natural-sft-s",
path_in_repo="d24_matformer_h16_natural_sft_s",
model_tag="d24_matformer_h16_natural_sft_s",
step=485,
base_dir="/tmp/nanochat-eval",
)
Then run eval with:
NANOCHAT_BASE_DIR=/tmp/nanochat-eval python -m scripts.base_eval --model-tag d24_matformer_h16_natural_sft_s --step 485