maum-ai_-_Llama-3-MAAL-8B-Instruct-v0.1-8bits

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Llama-3-MAAL-8B-Instruct-v0.1 - bnb 8bits

• Model creator: https://huggingface.co/maum-ai/
• Original model: https://huggingface.co/maum-ai/Llama-3-MAAL-8B-Instruct-v0.1/

Original model description:

license: llama3 base_model:

• meta-llama/Meta-Llama-3-8B-Instruct language:
• en
• ko tags:
• facebook
• meta
• llama
• llama-3
• llama-3-ko

---

<p align="left"> <img src="https://cdn-uploads.huggingface.co/production/uploads/646484cfb90150b2706df03b/BEOyMpnnY9VY2KXlc3V2F.png" width="20%"/> <p>

Llama-3-MAAL-8B-Instruct-v0.1

we release MAAL, Multilingual Adaptive Augmentation Language-model which comprises a groundbreaking fusion of multilingual capabilities and adaptive augmentation techniques.

• Developed by: maum.ai Brain NLP. Jaeyoon Jung, Jinjoo Lee, Yongjae Lee, Dongjun Lee, Woosung Joo
• Language(s) (NLP): Korean, English (currently, bilingual)

Model Description

Version 0.1 uses cross-lingual training to transfer instruction-following capabilities from English to Korean.

• We Trained this model on an 8 H100-80G for 1 day with cross-lingual training dataset
• we recommend using the fixed system prompt for the model unless you fine-tune it

너는 마음에이아이의 챗봇 MAAL이다. 고객의 질문에 친절하게 답하여라.

sample inference code (GPU)

import transformers
import torch

model_id = "maum-ai/Llama-3-MAAL-8B-Instruct-v0.1"
model = transformers.AutoModelForCausalLM.from_pretrained(model_id).to("cuda")
tokenizer = transformers.AutoTokenizer.from_pretrained(model_id)
streamer = transformers.TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)

# we recommend using the fixed prompt for the model unless you fine-tune it
prompt = "너는 마음에이아이의 챗봇 MAAL이다. 고객의 질문에 친절하게 답하여라."
instruction = "사과 한 박스에는 사과가 30개 들어있는데, 처음에는 사과 3박스가 있었고, 내가 사과 5개를 먹었어. 남은 사과는 총 몇개야?"

messages = [
    {"role": "system", "content": f"{prompt}"},
    {"role": "user", "content": f"{instruction}"}
    ]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    return_tensors='pt').to("cuda")
outputs = model.generate(inputs, streamer=streamer, max_new_tokens=1024, pad_token_id=tokenizer.eos_token_id)

Evaluation Results

As the main goal of version 0.1 is to transfer instruction-following capabilities from English to Korean without utilizing continuous pre-training, etc., we select LogicKor as our evaluation method to assess the Korean instruction skills.

We compare our model with a similar parameter model (less than 13B) that has been fine-tuned on the Korean dataset. * denotes our self-report result.

Limitations

Due to this model being trained on a small dataset, it has several limitations.

• Hard to generate diverse Korean texts
• lack of Korean knowledge & Culture (localization)
• Not work with Image inputs and video inputs

Todo

we will solve these limitations one by one by upgrading this model like as...

• Enhance the Korean generation through Vocabulary Expansion & Continuous pre-training. (more Korean corpus!)
• Localize with cultural adaptation method and additional Korean knowledge data. similar idea
• Develop a Vision Language Model that can handle both video and image inputs. similar idea