tibeb-sft-adapter

Tibeb SFT Adapter

LoRA adapters for Tibeb AI — Ethiopia's Amharic financial assistant, fine-tuned on Aya Expanse 8B.

Model Description

Tibeb is an Amharic-language AI assistant focused on financial literacy in the Ethiopian context. These adapters were trained using MLX LoRA on Apple Silicon.

Training Details

• Base model: CohereForAI/aya-expanse-8b (4-bit quantized via mlx-community)
• Method: LoRA (rank 16, all layers, cosine LR decay, AdamW)
• Data: ~692K rows of Amharic instruction data (see dataset repo)
• Hardware: Apple Silicon Mac (MLX backend)

v3 Config

• Iterations: 20,000
• Batch size: 1 (4x gradient accumulation = effective batch 4)
• Learning rate: 1e-5 → 1e-6 (cosine decay, 500 warmup steps)
• LoRA rank: 16, dropout: 0.05, scale: 32.0
• Max sequence length: 512

Adapters

• mlx-adapter-v3/ — v3 training (rank 16, all layers, cosine LR)
• mlx-adapter/ — v2 training (rank 8, 16 layers)

Usage

from mlx_lm import load, generate

model, tokenizer = load(
    "mlx-community/aya-expanse-8b-4bit",
    adapter_path="nahommohan/tibeb-sft-adapter/mlx-adapter-v3"
)

prompt = "ስለ ቲ-ቢል ኢንቨስትመንት ንገረኝ"
response = generate(model, tokenizer, prompt=prompt, max_tokens=500)
print(response)

Links

• Training pipeline: github.com/nahomar/tibeb-training
• Dataset: nahommohan/tibeb-training-data