Downloads · 30 days
0
project-1746-LLaMA-8B-rucola-funetune
bruhtus/project-1746-LLaMA-8B-rucola-funetune
project-1746-LLaMA-8B-rucola-funetune is a machine learning model from bruhtus. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Model for my University project. Tried to do better than ruRoBERTa-large by using LLaMA 3 8B base and finetuning it but no such luck. I was inexperienced and short on time but my LoRA was getting steadily better, so i…
Access
Public
Updated May 14, 2024
Repo size
1.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.3 GB · 99%
At a glance
- Access
- Public
- Created
- May 14, 2024
- Updated
- May 14, 2024
- SHA
- 517c8036
Datasets
Model card
From the Hugging Face model README
Model for my University project. Tried to do better than ruRoBERTa-large by using LLaMA 3 8B base and finetuning it but no such luck. I was inexperienced and short on time but my LoRA was getting steadily better, so it might be possible.
The idea behind my finetuning was that a model trained on such large number of tokens would understand nuances of russian language much better and would need only small guidance that could be provided by finetuning on rucola dataset.
Final numbers for accuracy and mcc were 0.742 and 0.419 respectively. Not great not terrible: it is in TOP20 of this leaderboard https://rucola-benchmark.com/leaderboard
- Created
- May 14, 2024
- Updated
- May 14, 2024