Ntahokaja-0.31-Base

Technical Report: Ntahokaja 0.31 (Base)

Ntahokaja 0.31 is the first sovereign Small Language Model (SLM) trained from scratch specifically for the Kirundi language. Developed by Nyehashi AI, it features a custom morphological tokenizer designed to respect Bantu grammar and agglutinative logic.

Quick Start (Usage)

This model requires custom tokenization logic. You must use trust_remote_code=True to load it correctly.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Nyehashai/Ntahokaja-0.31-Base" 

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "Amategeko yu'Burundi avuga ko "
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_length=100, do_sample=True, temperature=0.7)

print(tokenizer.decode(outputs[0]))

Section 1 — Executive Summary & Architecture

Ntahokaja 0.31 is a 91,209,216-parameter Transformer-based language model. Unlike general models, it uses a Hybrid Morphological-BPE strategy. This version resolves the "word-boundary collapse" by using an explicit @@ sub-word joiner and ByteLevel space protection. Layers: 12 Attention Heads: 12 Context Window: 512 tokens Size: 347.9 MB (Safetensors)

Section 2 — Performance

Metric Value Validation Loss 3.24 (Optimal) Training Loss 1.26 Vocabulary Size 7,500 tokens Data Volume 18.1M tokens

Section 3 — Behavioral Observations

The model demonstrates high-fidelity knowledge of Burundian geography (identifying provinces like Makamba, Kirundo, etc.) and political history. Notably, the model exhibits Implicit Moral Alignment, pivoting away from harmful or violent prompts toward themes of social respect and constitutional law.

Section 4 — Intended Use & Limitations

This is a Foundational Base Model. It is designed for text continuation, linguistic research, and as a base for instruction fine-tuning. It is NOT a conversational assistant and has not been fine-tuned for QA yet.

Citation and Attribution

If you use this model, please attribute the work to Nyehashi AI:

Lead Engineer: Ishimwe Lilas

Additive Current Data : Iradukunda Cherina, Itegineza Alvin

Economic Advisory: Igiraneza Charine

Organization: Nyehashi AI (Burundi)