Downloads · 30 days
4
22% of all-time downloads
rost-73m-base
rostlabs/rost-73m-base
rost-73m-base is a machine learning model from rostlabs. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc-by-nc-4.0.
the pretrained base model, depth d6, checkpoint step 000708.
All-time downloads
18
Public
Repo size
581 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt194 MB · 100%
At a glance
- License
- cc-by-nc-4.0
- Model type
- nanochat-gpt
- Access
- Public
- Created
- Jul 30, 2026
- Updated
- Aug 2, 2026
- SHA
- e7948f54
Model card
From the Hugging Face model README
rostlabs/rost-d6-base
the pretrained base model, depth d6, checkpoint step 000708.
Trained from scratch on a bilingual Romanian/English mixture. This is a research model at GPT-2 scale, not a product.
| architecture | nanochat GPT, d6, 2048 context |
| tokenizer | rostlabs/rost-tok-bilingual, 32,768 vocab, bilingual |
| pretraining mixture | 30% Romanian / 70% English, interleaved at document level |
| Romanian source | FineWeb2-ro, educational score >= 3, diacritic-normalized |
| English source | ClimbMix |
| stage | base |
The tokenizer is included under tokenizer/ so this repo is self-contained.
Licence
CC-BY-NC-4.0, non-commercial. Both halves of the training data carry non-commercial terms: ClimbMix is CC-BY-NC-4.0 ("for research and development only"), and the OpenLLM-Ro SFT datasets are CC-BY-NC-4.0. The model inherits that. Note that the ClimbMix mirror used is tagged MIT, which does not override the upstream terms.
Known characteristics
The Romanian pretraining data is roughly one third health content by character count (measured across three shards of the source), with history, geography, finance and education each near 10%. Entertainment, gaming and software are each under 1%. Expect fluency to be uneven across domains accordingly.
Reproduce
bash runs/speedrun_ro.sh
- Type
- nanochat-gpt
- License
- cc-by-nc-4.0
- Languages
- ro, en
- Created
- Jul 30, 2026
- Updated
- Aug 2, 2026