Llama-3.2-Korean-GGACHI-1B-Instruct-v1

@ 2024.10.07 Model torchtorchkimtorch/Llama-3.2-Korean-GGACHI-1B-Instruct-v1 Released!

@ 2024.10.18 Performance for KOBEST of Llama-3.2-Korean-GGACHI-1B-Instruct-v1 has been updated!

Llama-3.2-Korean-GGACHI-1B-Instruct-v1

모델 설명 (Model Description)

GGACHI-1B-Instruct-v1는 Llama-3.2-1B-Instruct 모델을 기반으로 하는 한국어 태스크 수행에 최적화된 instruction-tuned 언어 모델입니다. 230,000개 이상의 고품질 한국어 데이터셋을 사용하여 fine-tuning되었습니다.

GGACHI-1B-Instruct-v1 is an instruction-tuned language model optimized for Korean language tasks, based on the Llama-3.2-1B-Instruct model. It has been fine-tuned using over 230,000 high-quality Korean language datasets.

모델 성능 (Model Performance)

- 0 shot

<table style="width:100%; text-align:center; border-collapse:collapse;"> <thead> <tr> <th style="border:1px solid black;">Task</th> <th style="border:1px solid black;">Model</th> <th style="border:1px solid black;">Accuracy</th> </tr> </thead> <tbody> <tr> <td rowspan="2" style="border:1px solid black;">kobest_boolq</td> <td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> <td style="border:1px solid black;"><strong>0.502</td> </tr> <tr> <td style="border:1px solid black;"><strong>GGACHI</strong></td> <td style="border:1px solid black;"><strong>0.502</td> </tr> <tr> <td rowspan="2" style="border:1px solid black;">kobest_copa</td> <td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> <td style="border:1px solid black;">0.504</td> </tr> <tr> <td style="border:1px solid black;"><strong>GGACHI</strong></td> <td style="border:1px solid black;"><strong>0.521</strong></td> </tr> <tr> <td rowspan="2" style="border:1px solid black;">kobest_hellaswag</td> <td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> <td style="border:1px solid black;">0.358</td> </tr> <tr> <td style="border:1px solid black;"><strong>GGACHI</strong></td> <td style="border:1px solid black;"><strong>0.380</td> </tr> <tr> <td rowspan="2" style="border:1px solid black;">kobest_sentineg</td> <td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> <td style="border:1px solid black;">0.476</td> </tr> <tr> <td style="border:1px solid black;"><strong>GGACHI</strong></td> <td style="border:1px solid black;"><strong>0.594</strong></td> </tr> </tbody> </table>

- 5 shot

<table style="width:100%; text-align:center; border-collapse:collapse;"> <thead> <tr> <th style="border:1px solid black;">Task</th> <th style="border:1px solid black;">Model</th> <th style="border:1px solid black;">Accuracy</th> </tr> </thead> <tbody> <tr> <td rowspan="2" style="border:1px solid black;">kobest_boolq</td> <td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> <td style="border:1px solid black;"><strong>0.571</td> </tr> <tr> <td style="border:1px solid black;"><strong>GGACHI</strong></td> <td style="border:1px solid black;">0.565</td> </tr> <tr> <td rowspan="2" style="border:1px solid black;">kobest_copa</td> <td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> <td style="border:1px solid black;">0.526</td> </tr> <tr> <td style="border:1px solid black;"><strong>GGACHI</strong></td> <td style="border:1px solid black;"><strong>0.549</strong></td> </tr> <tr> <td rowspan="2" style="border:1px solid black;">kobest_hellaswag</td> <td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> <td style="border:1px solid black;">0.364</td> </tr> <tr> <td style="border:1px solid black;"><strong>GGACHI</strong></td> <td style="border:1px solid black;"><strong>0.398</td> </tr> <tr> <td rowspan="2" style="border:1px solid black;">kobest_sentineg</td> <td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> <td style="border:1px solid black;">0.725</td> </tr> <tr> <td style="border:1px solid black;"><strong>GGACHI</strong></td> <td style="border:1px solid black;"><strong>0.795</strong></td> </tr> </tbody> </table>

- 10 shot

<table style="width:100%; text-align:center; border-collapse:collapse;"> <thead> <tr> <th style="border:1px solid black;">Task</th> <th style="border:1px solid black;">Model</th> <th style="border:1px solid black;">Accuracy</th> </tr> </thead> <tbody> <tr> <td rowspan="2" style="border:1px solid black;">kobest_boolq</td> <td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> <td style="border:1px solid black;"><strong>0.593</td> </tr> <tr> <td style="border:1px solid black;"><strong>GGACHI</strong></td> <td style="border:1px solid black;">0.571</td> </tr> <tr> <td rowspan="2" style="border:1px solid black;">kobest_copa</td> <td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> <td style="border:1px solid black;">0.525</td> </tr> <tr> <td style="border:1px solid black;"><strong>GGACHI</strong></td> <td style="border:1px solid black;"><strong>0.549</strong></td> </tr> <tr> <td rowspan="2" style="border:1px solid black;">kobest_hellaswag</td> <td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> <td style="border:1px solid black;">0.356</td> </tr> <tr> <td style="border:1px solid black;"><strong>GGACHI</strong></td> <td style="border:1px solid black;"><strong>0.394</td> </tr> <tr> <td rowspan="2" style="border:1px solid black;">kobest_sentineg</td> <td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> <td style="border:1px solid black;">0.768</td> </tr> <tr> <td style="border:1px solid black;"><strong>GGACHI</strong></td> <td style="border:1px solid black;"><strong>0.821</strong></td> </tr> </tbody> </table>

Contact

• 김민혁(Minhyuk Kim)
  Mail: [email protected]
  LinkedIn : https://www.linkedin.com/in/mhkim0929/