Downloads · 30 days
0
nanogpt-tr-v5-ckpts
musabc/nanogpt-tr-v5-ckpts
nanogpt-tr-v5-ckpts is a machine learning model from musabc. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
V5 200M Türkçe pretrained LM, multi-stage curriculum eğitimi.
Access
Public
Updated May 16, 2026
Repo size
30.2 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt3.6 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- May 12, 2026
- Updated
- May 16, 2026
- SHA
- 8882ab81
Model card
From the Hugging Face model README
nanogpt-tr-v5 Checkpoints
V5 200M Türkçe pretrained LM, multi-stage curriculum eğitimi.
Mimari
- 18 layer, 14 head, 896 embd
- 32K vocab, 2048 context
- RoPE (theta=100K) + RMSNorm + SwiGLU + QK-norm
- Logit soft-cap (30) + tied embeddings
- 210M parametre
Eğitim
- 21.6B token, multi-stage curriculum (web → medium → premium annealing)
- Muon (2D weights) + AdamW (1D + embed)
- bf16 mixed precision, torch.compile
- Lightning AI → Thunder Compute migration
Yükleme
import torch
from model_v5 import GPTV5, GPTConfigV5
ckpt = torch.load("best_ckpt.pt", weights_only=False)
cfg = GPTConfigV5(**ckpt["config"])
model = GPTV5(cfg)
state = {k.replace("_orig_mod.", ""): v for k, v in ckpt["model"].items()}
model.load_state_dict(state)
Sample
code repo'sundaki 06_sample.py kullanın.
- License
- apache-2.0
- Languages
- tr
- Created
- May 12, 2026
- Updated
- May 16, 2026