Downloads · 30 days
37
1% of all-time downloads
ruadapt_llama3_8b_instruct_extended_lep_ft
RefalMachine/ruadapt_llama3_8b_instruct_extended_lep_ft
ruadapt_llama3_8b_instruct_extended_lep_ft is a machine learning model from RefalMachine. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
LoRa tuned version of ruadapt llama 3 8B with extended tokenizer after LEP (Learned Embedding Propagation, paper will be soon) procedure on saigascored d7 dataset.
All-time downloads
2.7K
Public
Parameters
8.4B
16.8 GB on disk
Likes
4
Public
Repo makeup
Click a slice to open those files.
.safetensors16.8 GB · 100%
At a glance
- Model type
- llama
- Access
- Public
- Created
- Aug 22, 2024
- Updated
- Aug 22, 2024
- SHA
- 0bbab0f6
Base models
Datasets
Model card
From the Hugging Face model README
Model description
LoRa tuned version of ruadapt llama 3 8B with extended tokenizer after LEP (Learned Embedding Propagation, paper will be soon) procedure on saiga_scored d7 dataset.
Thanks to the extended tokenizer, the model works more efficiently with the Russian language.
How to cite:
Tikhomirov M., Chernyshev D. Facilitating large language model Russian adaptation with Learned Embedding Propagation // 2024 (will be soon)
Tikhomirov M., Chernyshev D. Impact of Tokenization on LLaMa Russian Adaptation //2023 Ivannikov Ispras Open Conference (ISPRAS). – IEEE, 2023. – С. 163-168.
- Type
- llama
- Languages
- ru, en
- Created
- Aug 22, 2024
- Updated
- Aug 22, 2024