Downloads · 30 days
62
1% of all-time downloads
africa-intelligence/llama-8b-south-africa
llama-8b-south-africa is a machine learning model from africa-intelligence. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
This model card provides detailed information about the LLaMA-3.1-8B model fine-tuned for South African languages. The model demonstrates cost-effective cross-lingual transfer learning for African language processing.
Downloads · 30 days
62
1% of all-time downloads
All-time downloads
5.2K
Public
Repo size
26 GB
Likes
3
Public
Click a slice to open those files.
.safetensors168 MB · 95%
From the Hugging Face model README
This model card provides detailed information about the LLaMA-3.1-8B model fine-tuned for South African languages. The model demonstrates cost-effective cross-lingual transfer learning for African language processing.
The model is based on Meta's LLaMA-3.1-8B-Instruct architecture and has been fine-tuned on translated versions of the Alpaca Cleaned dataset. The training approach leverages machine translation to create instruction-tuning data in five South African languages, making it a cost-effective solution for multilingual AI development.
The training data was created by translating the Alpaca Cleaned dataset into five target languages:
Machine translation was used to generate the training data, with a cost of $370 per language.
The model was trained using the PEFT (Parameter-Efficient Fine-Tuning) library on the Akash Compute Network. Key aspects of the training process include:
Current evaluation metrics are available for two languages:
Evaluation was conducted using three benchmark datasets:
Evaluation Coverage
Training Data Quality
Performance Gaps
The model requires the following framework versions:
from transformers import AutoModelForCausalLM, AutoTokenizer
# Load the model and tokenizer
model_name = "meta-llama/llama-8b-south-africa"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
# Example usage for text generation
text = "Translate to Xhosa: Hello, how are you?"
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_length=50)
result = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(result)
This model is released under the Apache 2.0 license. The full license text can be found at https://www.apache.org/licenses/LICENSE-2.0.txt