Downloads · 30 days
1.1K
7% of all-time downloads
RichardErkhov/MaLA-LM_-_emma-500-llama2-7b-gguf
MaLA-LM_-_emma-500-llama2-7b-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
1.1K
7% of all-time downloads
All-time downloads
17.2K
Public
Repo size
76.6 GB
Likes
0
Public
Click a slice to open those files.
.gguf88.7 GB · 100%
From the Hugging Face model README
Quantization made by Richard Erkhov.
emma-500-llama2-7b - GGUF
| Name | Quant method | Size |
|---|---|---|
| emma-500-llama2-7b.Q2_K.gguf | Q2_K | 2.36GB |
| emma-500-llama2-7b.IQ3_XS.gguf | IQ3_XS | 2.6GB |
| emma-500-llama2-7b.IQ3_S.gguf | IQ3_S | 2.75GB |
| emma-500-llama2-7b.Q3_K_S.gguf | Q3_K_S | 2.75GB |
| emma-500-llama2-7b.IQ3_M.gguf | IQ3_M | 2.9GB |
| emma-500-llama2-7b.Q3_K.gguf | Q3_K | 3.07GB |
| emma-500-llama2-7b.Q3_K_M.gguf | Q3_K_M | 3.07GB |
| emma-500-llama2-7b.Q3_K_L.gguf | Q3_K_L | 3.35GB |
| emma-500-llama2-7b.IQ4_XS.gguf | IQ4_XS | 3.4GB |
| emma-500-llama2-7b.Q4_0.gguf | Q4_0 | 3.56GB |
| emma-500-llama2-7b.IQ4_NL.gguf | IQ4_NL | 3.58GB |
| emma-500-llama2-7b.Q4_K_S.gguf | Q4_K_S | 3.59GB |
| emma-500-llama2-7b.Q4_K.gguf | Q4_K | 3.8GB |
| emma-500-llama2-7b.Q4_K_M.gguf | Q4_K_M | 3.8GB |
| emma-500-llama2-7b.Q4_1.gguf | Q4_1 | 3.95GB |
| emma-500-llama2-7b.Q5_0.gguf | Q5_0 | 4.33GB |
| emma-500-llama2-7b.Q5_K_S.gguf | Q5_K_S | 4.33GB |
| emma-500-llama2-7b.Q5_K.gguf | Q5_K | 4.45GB |
| emma-500-llama2-7b.Q5_K_M.gguf | Q5_K_M | 4.45GB |
| emma-500-llama2-7b.Q5_1.gguf | Q5_1 | 4.72GB |
| emma-500-llama2-7b.Q6_K.gguf | Q6_K | 5.15GB |
| emma-500-llama2-7b.Q8_0.gguf | Q8_0 | 6.67GB |
license: llama2 datasets:
EMMA-500 is a state-of-the-art multilingual language model designed to improve language representation, especially in low-resource languages, through continual pre-training on the Llama 2 7B architecture. Leveraging the MaLA Corpus, which spans over 500 languages and 74 billion tokens, EMMA-500 excels in multilingual tasks like commonsense reasoning, machine translation, open-ended generation, and text classification.
EMMA-500 outperforms other Llama 2-based models in diverse multilingual settings while maintaining robustness in specialized tasks.
You can use EMMA-500 for multilingual text generation. Below is an example to generate text using the model:
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "MaLA-LM/emma-500-llama2-7b"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
input_text = "Once upon a time"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
EMMA-500 was evaluated across multiple benchmarks and tasks, demonstrating:
Challenges remain in low-resource languages, where the model tends to have higher Self-BLEU scores, indicating reduced output diversity.
@article{ji2024emma500enhancingmassivelymultilingual,
title={{EMMA}-500: Enhancing Massively Multilingual Adaptation of Large Language Models},
author={Shaoxiong Ji and Zihao Li and Indraneil Paul and Jaakko Paavola and Peiqin Lin and Pinzhen Chen and Dayyán O'Brien and Hengyu Luo and Hinrich Schütze and Jörg Tiedemann and Barry Haddow},
year={2024},
journal={arXiv preprint 2409.17892},
url={https://arxiv.org/abs/2409.17892},
}
We extend our thanks to the language communities and contributors who helped source, clean, and validate the diverse data used in the MaLA Corpus. Their efforts are invaluable in supporting linguistic diversity in AI research.
This work is done by researchers at Helsinki-NLP in collaboration with partners from TU Darmstadt, the University of Edinburgh, and LMU Munich. It is funded by HPLT and UTTER.