Downloads · 30 days
28
2% of all-time downloads
Llama-2-7b-Ukrainian
tartuNLP/Llama-2-7b-Ukrainian
Llama-2-7b-Ukrainian is a text generation model from tartuNLP. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama2.
Llama-2-7b-Ukrainian is a bilingual pre-trained model supporting Ukrainian and English. Continued pre-training from Llama-2-7b on 5B tokens consisting of 75% Ukrainian documents and 25% English documents from CulturaX.
All-time downloads
1.5K
Public
Parameters
6.7B
13.5 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors13.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama2
- Model type
- llama
- Access
- Public
- Created
- May 29, 2024
- Updated
- Feb 8, 2026
- SHA
- b0634fbc
Base models
Datasets
Model card
From the Hugging Face model README
Llama-2-7b-Ukrainian
Model Details
Model Description
Llama-2-7b-Ukrainian is a bilingual pre-trained model supporting Ukrainian and English. Continued pre-training from Llama-2-7b on 5B tokens consisting of 75% Ukrainian documents and 25% English documents from CulturaX.
Paper: To Err Is Human, but Llamas Can Learn It Too
Training Hyperparameters
| Hyperparameter | Value |
|---|---|
| Training steps | 19080 |
| Batch size | 256 |
| Weight decay | 0.1 |
| Context length | 1024 |
| Learning rate | 2e-5 linear decay to 2e-6 |
| Precision | bf16 |
| Optimizer | AdamW |
Citation
BibTeX:
@article{luhtaru2024err,
title={To Err Is Human, but Llamas Can Learn It Too},
author={Luhtaru, Agnes and Purason, Taido and Vainikko, Martin and Del, Maksym and Fishel, Mark},
journal={arXiv preprint arXiv:2403.05493},
year={2024}
}
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama2
- Languages
- uk, en
- Created
- May 29, 2024
- Updated
- Feb 8, 2026