Downloads · 30 days
0
luminaudoabba/llama3-naijaweb-merged
llama3-naijaweb-merged is a machine learning model from luminaudoabba. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
This repository contains the LoRA adapter weights for NaijaLLaMA-8B, a parameter-efficient fine-tuned variant of LLaMA-3.1-8B adapted for Nigerian English and Nigerian Pidgin contexts.
Downloads · 30 days
0
Access
Public
Updated Jan 26, 2026
Repo size
235 MB
Likes
0
Public
Click a slice to open those files.
.safetensors218 MB · 93%
From the Hugging Face model README
This repository contains the LoRA adapter weights for NaijaLLaMA-8B, a parameter-efficient fine-tuned variant of LLaMA-3.1-8B adapted for Nigerian English and Nigerian Pidgin contexts.
The base model used was:
unsloth/llama-3.1-8b-bnb-4bit
Fine-tuning was performed using QLoRA via the Unsloth framework and Hugging Face TRL.
⚠️ This repository contains LoRA adapter weights only, not the full merged model.
Large language models are predominantly trained on Western-centric corpora and may underperform on culturally grounded Nigerian expressions and Pidgin syntax.
This work investigates whether conservative, low-rank QLoRA adaptation on a small Nigerian corpus (12,000 samples) can induce measurable contextual alignment under strict hardware constraints (single Tesla T4, 16GB VRAM).
This model accompanies the paper:
Low-resource Fine-Tuning of LLaMA-3.1 using QLoRA for Nigerian Language Contexts
saheedniyi/naijawebq_projk_projv_projo_projTraining loss decreased from approximately 2.04 to 1.98 over one epoch, indicating stable optimization under conservative scaling.
Perplexity on the held-out validation subset reduced from 8.20 (base) to 7.38 (fine-tuned).
Note: This model is an exploratory academic artifact. Improvements are modest and primarily observed in culturally grounded prompts.
Load the base model and attach the adapters:
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="unsloth/llama-3.1-8b-bnb-4bit",
max_seq_length=1024,
load_in_4bit=True,
)
model.load_adapter("luminaudoabba/llama3-naijaweb-merged")
model.enable_adapter_layers()