Downloads · 30 days
28
0% of all-time downloads
Llama-3-Ko-8B-OpenOrca
werty1248/Llama-3-Ko-8B-OpenOrca
Llama-3-Ko-8B-OpenOrca is a text generation model from werty1248. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.
Original model: beomi/Llama-3-Open-Ko-8B (2024.04.24 버전)
All-time downloads
30.5K
Public
Parameters
8B
48.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors16.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama3
- Model type
- llama
- Access
- Public
- Created
- Apr 27, 2024
- Updated
- May 20, 2024
- SHA
- ac40a26e
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
Llama-3-Ko-OpenOrca
<!-- Provide a quick summary of what the model is/does. -->Model Details
Model Description
<!-- Provide a longer summary of what this model is. -->Original model: beomi/Llama-3-Open-Ko-8B (2024.04.24 버전)
Dataset: kyujinpy/OpenOrca-KO
Training details
Training: Axolotl을 이용해 LoRA-8bit로 4epoch 학습 시켰습니다.
- sequence_len: 4096
- bf16
학습 시간: A6000x2, 6시간
Evaluation
- 0 shot kobest
| Tasks | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|
| kobest_boolq | 0 | acc | 0.5021 | ± | 0.0133 |
| kobest_copa | 0 | acc | 0.6920 | ± | 0.0146 |
| kobest_hellaswag | 0 | acc | 0.4520 | ± | 0.0223 |
| kobest_sentineg | 0 | acc | 0.7330 | ± | 0.0222 |
| kobest_wic | 0 | acc | 0.4881 | ± | 0.0141 |
- 5 shot kobest
| Tasks | n-shot | Metric | Value | Stderr | |
|---|---|---|---|---|---|
| kobest_boolq | 5 | acc | 0.7123 | ± | 0.0121 |
| kobest_copa | 5 | acc | 0.7620 | ± | 0.0135 |
| kobest_hellaswag | 5 | acc | 0.4780 | ± | 0.0224 |
| kobest_sentineg | 5 | acc | 0.9446 | ± | 0.0115 |
| kobest_wic | 5 | acc | 0.6103 | ± | 0.0137 |
License:
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama3
- Created
- Apr 27, 2024
- Updated
- May 20, 2024