Downloads · 30 days
0
nanochat-d20
shu127/nanochat-d20
nanochat-d20 is a machine learning model from shu127. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
This is a d20 (20-layer, 561M parameter) ChatGPT-like model trained using the nanochat pipeline.
Access
Public
Updated Oct 14, 2025
Repo size
2.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt2.1 GB · 100%
At a glance
- License
- mit
- Access
- Public
- Created
- Oct 14, 2025
- Updated
- Oct 14, 2025
- SHA
- 824021ff
Model card
From the Hugging Face model README
NanoChat d20 Model
This is a d20 (20-layer, 561M parameter) ChatGPT-like model trained using the nanochat pipeline.
Model Details
- Parameters: 560,988,160
- Layers: 20
- Hidden dimension: 1280
- Attention heads: 10
- Vocabulary size: 65,536
- Context length: 2048
Training
- BASE stage: FineWeb-Edu pretraining
- MID stage: SmolTalk + MMLU + GSM8K
- SFT stage: ARC + GSM8K + SmolTalk
- License
- mit
- Languages
- en
- Created
- Oct 14, 2025
- Updated
- Oct 14, 2025