Downloads · 30 days
5
22% of all-time downloads
rost-73m-sft
rostlabs/rost-73m-sft
rost-73m-sft is a machine learning model from rostlabs. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc-by-nc-4.0.
the instruction-tuned chat model, depth d6, checkpoint step 000040.
All-time downloads
23
Public
Repo size
581 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt387 MB · 100%
At a glance
- License
- cc-by-nc-4.0
- Model type
- nanochat-gpt
- Access
- Public
- Created
- Jul 30, 2026
- Updated
- Aug 2, 2026
- SHA
- 77390ef3
Model card
From the Hugging Face model README
rostlabs/rost-d6-sft
the instruction-tuned chat model, depth d6, checkpoint step 000040.
Trained from scratch on a bilingual Romanian/English mixture. This is a research model at GPT-2 scale, not a product.
| architecture | nanochat GPT, d6, 2048 context |
| tokenizer | rostlabs/rost-tok-bilingual, 32,768 vocab, bilingual |
| pretraining mixture | 30% Romanian / 70% English, interleaved at document level |
| Romanian source | FineWeb2-ro, educational score >= 3, diacritic-normalized |
| English source | ClimbMix |
| stage | sft |
The tokenizer is included under tokenizer/ so this repo is self-contained.
Licence
CC-BY-NC-4.0, non-commercial. Both halves of the training data carry non-commercial terms: ClimbMix is CC-BY-NC-4.0 ("for research and development only"), and the OpenLLM-Ro SFT datasets are CC-BY-NC-4.0. The model inherits that. Note that the ClimbMix mirror used is tagged MIT, which does not override the upstream terms.
Known characteristics
The Romanian pretraining data is roughly one third health content by character count (measured across three shards of the source), with history, geography, finance and education each near 10%. Entertainment, gaming and software are each under 1%. Expect fluency to be uneven across domains accordingly.
Reproduce
bash runs/speedrun_ro.sh
- Type
- nanochat-gpt
- License
- cc-by-nc-4.0
- Languages
- ro, en
- Created
- Jul 30, 2026
- Updated
- Aug 2, 2026