Downloads · 30 days
10
17% of all-time downloads
ruadapt_mistral_7b_openchat_extended_lep_ft
RefalMachine/ruadapt_mistral_7b_openchat_extended_lep_ft
ruadapt_mistral_7b_openchat_extended_lep_ft is a machine learning model from RefalMachine. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
LoRa tuned version of ruadapt mistral 7B v0.1 with extended tokenizer after LEP (Learned Embedding Propagation, paper will be soon) procedure on saigascored d7 dataset.
All-time downloads
60
Public
Parameters
7.4B
14.9 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors14.9 GB · 100%
At a glance
- Model type
- mistral
- Access
- Public
- Created
- Aug 22, 2024
- Updated
- Aug 22, 2024
- SHA
- 52ec0e1a
Base models
Datasets
Model card
From the Hugging Face model README
Model description
LoRa tuned version of ruadapt mistral 7B v0.1 with extended tokenizer after LEP (Learned Embedding Propagation, paper will be soon) procedure on saiga_scored d7 dataset.
Thanks to the extended tokenizer, the model works more efficiently with the Russian language.
How to cite:
Tikhomirov M., Chernyshev D. Facilitating large language model Russian adaptation with Learned Embedding Propagation // 2024 (will be soon)
Tikhomirov M., Chernyshev D. Impact of Tokenization on LLaMa Russian Adaptation //2023 Ivannikov Ispras Open Conference (ISPRAS). – IEEE, 2023. – С. 163-168.
- Type
- mistral
- Languages
- ru, en
- Created
- Aug 22, 2024
- Updated
- Aug 22, 2024