Downloads · 30 days
174
12% of all-time downloads
chrono-2021-sn38
kepton0117/chrono-2021-sn38
chrono-2021-sn38 is a text generation model from kepton0117. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
Continue-pretrained sn38-nanochrono (~2.02B) for Bittensor subnet 38, cutoff 2021-12-31.
All-time downloads
1.5K
Public
Parameters
2B
4 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors4 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- other
- Model type
- sn38-nanochrono
- Access
- Public
- Created
- Sep 18, 2026
- Updated
- Sep 21, 2026
- SHA
- 0579b366
Try a prompt
Model card
From the Hugging Face model README
chrono-2021-sn38 — SN38 Round 12 (year 2021)
Continue-pretrained sn38-nanochrono (~2.02B) for Bittensor subnet 38, cutoff 2021-12-31.
- Pin (this repo):
f62e238265e3ca4a4ed20b300475513446d9e1fc= train step-4000 - Base:
anacoluthe89/chrono-2020@633eb832c3de8eceb0b5bba1c5fb94faa8e777c6(UID 131) - Data + scripts:
kepton0117/sn38-train-2021 - Code: github.com/kepton0117/sn38
- Cosine vs UID 131 at this pin: 0.9294 (copy gate needs < 0.95)
recipe/ in this repo is the train algorithm and launch scripts that produced the pin.
Algorithm
Full-parameter causal LM continue-pretrain (CPT). Not LoRA. Not instruction SFT.
loss = cross_entropy(model(tokens[:-1]), tokens[1:])
| Piece | Value |
|---|---|
| Arch | sn38-nanochrono, 28 layers, hidden 1792, seq 2048 |
| Update | all parameters |
| Replay | 15% FineWeb 2017–2020 |
| Packing | docs concatenated, cut every 2048 tokens, EOS between docs |
| Optimizer | AdamW, weight decay 0.1, betas (0.9, 0.95) |
| Schedule | cosine decay, 3% warmup |
| Parallel | 8× H200 DDP |
Recipe that produced this pin
| Knob | Value |
|---|---|
| GPUs | 8× H200 |
| Micro-batch / GPU | 16 |
| Grad accum | 2 |
| Tokens / step | 8 × 16 × 2048 × 2 = 524,288 |
| LR | 1e-3 (1e-4 / 3e-4 left cosine stuck at ~0.999) |
| Checkpoint | step-4000 |
| Data | FineWeb last-4 2021 dumps (60%) + older FineWeb (15%) + brimmann2/enwiki-dec2021 (25%) |
| Filter | drop short / nav / repetitive docs (~12.5%) |
| Packed tokens | 2,610,770 × 2048 ≈ 5.35B |
# needs sn38.architectures from the GitHub repo
LR=1e-3 bash scripts/run_train_h200x8_2021.sh
Do not use FineWeb name='default' or Wikipedia 20220301.en — those leak 2022+.
Files
| Path | What |
|---|---|
model.safetensors | step-4000 weights |
config.json | nanochrono config |
recipe/ | launcher, trainer, data build, filter |
- Task
- Text Generation
- Library
- transformers
- Type
- sn38-nanochrono
- License
- other
- Created
- Sep 18, 2026
- Updated
- Sep 21, 2026