vinallama-12.5b-chat-DUS

vilm/vinallama-12.5b-chat-DUS

vinallama-12.5b-chat-DUS is a text generation model from vilm. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.

Downloads · 30 days

244

10% of all-time downloads

All-time downloads

2.5K

Public

Parameters

12.5B

25 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors25 GB · 100%

At a glance

Task: Text Generation
Library: transformers
License: apache-2.0
Model type: llama
Access: Public
Created: Jan 4, 2024
Updated: Jan 11, 2024
SHA: ba40d8cb

Try a prompt

Model card

From the Hugging Face model README

VinaLLaMA - State-of-the-art Vietnamese LLMs

Read our Paper

Prompt Format (ChatML):

<|im_start|>system
Bạn là một trợ lí AI hữu ích. Hãy trả lời người dùng một cách chính xác.
<|im_end|>
<|im_start|>user
Hello world!<|im_end|>
<|im_start|>assistant

Evaluation

This table is copied from VBD-Llama2 with updated results from VinaLLaMA-12.5B-chat-DUS

Model	Model size	arc_vi (acc)	hellaswag_vi (acc)	mmlu_vi (acc)	truthfulqa_vi (acc)	Average
URA-LLaMA-13B	13B	0,3752	0,4830	0,3973	0,4574	0,4282
BLOOMZ-7B	7B	0,3205	0,4930	<b>0,3975</b>	0,4523	0,4158
PhoGPT-7B5-Instruct	7B	0,2470	0,2578	0,2413	0,4759	0,3055
SeaLLM-7B-chat	7B	0,3607	0,5112	0,3339	0,4948	0,4252
Vietcuna-7b-v3	7B	0,3419	0,4939	0,3354	0,4807	0,4130
VinaLLaMA-2.7B-chat	7B	0,3273	0,4814	0,3051	0,4972	0,4028
VinaLLaMA-7B-chat	7B	0,4239	0,5407	0,3932	0,5251	0,4707
VBD-LLaMA2-7B-50b	7B	0,3222	0,5195	0,2964	0,4614	0,3999
VBD-LLaMA2-7B-50b-Chat	7B	0,3585	0,5207	0,3444	0,5179	0,4354
<b>VinaLLaMA-12.5B-chat-DUS</b>	12.5B	<b>0,4325</b>	<b>0,5816</b>	0,3875	<b>0,5850</b>	<b>0,4967</b>

Merging Methods

This model is a merge of the following models made with LazyMergekit:

vilm/vinallama-7b-chat

🧩 Configuration

slices:
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [0, 16]
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [8, 16]
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [8, 16]      
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [16, 24]
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [16, 24]
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [24, 28]
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [24, 28]
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [28, 32]
merge_method: passthrough
dtype: bfloat16

Task: Text Generation
Library: transformers
Type: llama
License: apache-2.0
Created: Jan 4, 2024
Updated: Jan 11, 2024