k-gpt-v1

K-GPT v1 — Kenya's Sovereign AI

K-GPT is version one of Kenya's — and eventually Africa's — Sovereign AI project, being undertaken by a team from Univars.space under Rewel AI Labs.

K-GPT is designed to deeply understand and serve Kenyan citizens in their own languages, cultural contexts, legal frameworks, and economic realities. It is the first step toward building a truly African-grounded large language model.

---

Model Description

K-GPT v1 is a QLoRA-finetuned adaptation of Qwen 2.5 7B, trained on 27 curated Kenyan datasets totaling approximately 18.9 million tokens. The model specializes in Kenyan multilingual understanding, constitutional law, cultural nuance, and domain-specific knowledge across banking, tourism, agriculture, and governance.

Languages Supported

K-GPT v1 supports the following languages and dialects:

• Formal English (Kenyan context)
• Swahili (Standard & Kenyan)
• Sheng (Nairobi urban slang/pidgin)
• Kamba (Kikamba)
• Luo (Dholuo)
• Kikuyu (Gikuyu)
• Luhya (Maragoli & related)
• Kidawida (Taita)
• Kalenjin
• Somali (Kenyan Somali communities)

Specialized Knowledge Domains

• Kenyan Parliament Hansard — Constitutional debates and legislative proceedings
• 2010 Constitution of Kenya — Full legal text and interpretive context
• Mount Kenya Tourism Guide — Cultural and geographical tourism content
• Central Bank of Kenya (CBK) Regulations — Banking and financial compliance
• Kenyan Agricultural Data — Crop patterns, market prices, and farming guides
• Data Protection Act 2019 — Privacy and data sovereignty frameworks

---

Intended Uses & Limitations

Intended Uses

• Sovereign AI Chatbot: Powering culturally-aware conversational AI for Kenyan users via the Univars Sovereign Chat platform.
• Multilingual Customer Service: Serving Kenyans in their native languages across banking, government, tourism, and agriculture sectors.
• Legal & Constitutional Q&A: Answering questions about Kenyan law, the 2010 Constitution, and parliamentary proceedings.
• Cultural Preservation: Documenting and making accessible knowledge in underrepresented Kenyan languages (Sheng, Kamba, Kidawida, etc.).
• Research: Advancing African NLP and sovereign AI research.

Limitations

• Regional Focus: K-GPT v1 is primarily trained on Kenyan data. Performance on other African countries' contexts may be limited.
• Base Model Constraints: Inherits limitations of Qwen 2.5 7B, including potential biases in the pre-training data.
• Low-Resource Languages: While the model covers 10+ Kenyan languages, training data volume varies — Swahili and English have the deepest coverage, while languages like Kidawida and Somali have fewer examples.
• Not a Legal Advisor: While trained on constitutional and legal text, K-GPT should not be used as a substitute for professional legal advice.
• Hallucination Risk: Like all LLMs, K-GPT may generate plausible-sounding but incorrect information.

---

Training and Evaluation Data

Training Data

K-GPT v1 was trained on 27 carefully curated datasets covering:

Evaluation

• Training was monitored via loss curves across 1,041 steps.
• Final training loss: 0.000161 (indicating strong convergence).
• Qualitative evaluation was performed on multilingual prompts across all supported languages.
• The model is currently deployed in Sovereign Chat (Testing Studio) within the Univars LLMOps platform for ongoing human evaluation.

---

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B")
tokenizer = AutoTokenizer.from_pretrained("Univars/k-gpt-v1")

# Load K-GPT adapter
model = PeftModel.from_pretrained(base_model, "Univars/k-gpt-v1")

# Generate
inputs = tokenizer("Habari! Niambie kuhusu Katiba ya Kenya ya 2010.", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

About

K-GPT is built by the Univars team at Rewel AI Labs.

• Website: univars.space
• Organization: Rewel AI Labs
• Location: Nairobi, Kenya
• Access: Gated -- approval required

---

K-GPT v1 is the foundation for a pan-African Sovereign AI initiative. Future versions will expand to cover all 54 African nations, additional languages, and domain-specific capabilities.