mixtral-8x7b-hausa-lora-v1

Mixtral-8x7B Hausa LoRA v1

AutoScientist-Generated Adapter for Hausa Language

A LoRA adapter for Mixtral-8x7B fine-tuned on Hausa, generated autonomously by AdaptionLabs AutoScientist.

What This Is

An experimental research artifact exploring whether autonomous AI research systems can efficiently adapt large language models for low-resource African languages.

Despite 100M+ speakers, Hausa is severely underrepresented in language models. This adapter was produced by AutoScientist handling the entire pipeline: dataset selection (Dialectra Hausa Speech Corpus), preprocessing, training config, and execution.

Key point: This demonstrates AI-assisted language tech development for underrepresented communities.

Quick Start

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained(
    "mistralai/Mixtral-8x7B-Instruct-v0.1",
    load_in_4bit=True,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mixtral-8x7B-Instruct-v0.1")
model = PeftModel.from_pretrained(base_model, "k0ns0l/mixtral-8x7b-hausa-lora-v1")

# Generate
prompt = "Yaya kake?"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Hardware: 24GB+ GPU (4-bit quantization) or 80GB (full precision)

Training Details

Generated by AutoScientist:

• Dataset: Dialectra Hausa Speech Corpus v1 (speech transcriptions)
• LoRA: r=64, alpha=128, targets: q/k/v/o_proj
• Training: 5 epochs, 1,115 steps, bf16
• Loss: 1.65 → 0.83

Why Mixtral: MoE efficiency, 32k context, multilingual foundation, instruction-following

Status & Limitations

⚠️ Experimental - qualitative testing shows good Hausa generation, but comprehensive benchmarking incomplete.

Limitations:

• Primarily Nigerian Hausa dialect
• Training data from speech transcriptions (conversational style)
• Not evaluated on FLORES-200, MasakhaNER, or other benchmarks yet
• Inherits base model biases

Use responsibly: Validate outputs with native speakers, don't use for high-stakes decisions without oversight.

About AutoScientist

AutoScientist by AdaptionLabs automates ML research workflows—from experiment design to execution. Reported 35% improvement over human-configured pipelines.

This project demonstrates autonomous research for low-resource language adaptation.

Citation

@misc{k0ns0l2026mixtral-hausa,
  author = {k0ns0l},
  title = {Mixtral-8x7B Hausa LoRA v1: AutoScientist-Generated Adapter},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/k0ns0l/mixtral-8x7b-hausa-lora-v1}}
}

---

Contact: @k0ns0l | License: Apache 2.0 | Contributions welcome

Research artifact exploring autonomous AI for underrepresented language technology.