Downloads · 30 days
17
33% of all-time downloads
llama2-ko-DPO
colable/llama2-ko-DPO
llama2-ko-DPO is a text generation model from colable. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This is an Korean Model based on [beomi/open-llama-2-ko-7b]
All-time downloads
52
Public
Parameters
6.9B
13.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors13.7 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- llama
- Access
- Public
- Created
- Feb 5, 2024
- Updated
- Feb 5, 2024
- SHA
- 0a3bf586
Model card
From the Hugging Face model README
open-llama-2-ko based model with modified DPO dataset
This is an Korean Model based on
- [beomi/open-llama-2-ko-7b]
Dataset is modified from
- [SJ-Donald/orca-dpo-pairs-ko]
Parameters
learning_rate: float = 3e-4
lr_scheduler: str = "cosine"
warmup_ratio: float = 0.1
lora_r: int = 16
lora_alpha: int = 16
lora_dropout: float = 0.05
optim='paged_adamw_32bit'
bf16=True
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- mit
- Languages
- ko
- Created
- Feb 5, 2024
- Updated
- Feb 5, 2024