tekyerema-1-v2-tool-pilot

Tɛkyerɛma-1 tool adapter, v2 pilot (arm ①, sandwich English)

Author: Prince Nasamu Alhassan

Overview

Qwen3-4B + LoRA, trained on the v2 corpus to turn an utterance into one tool call. Arm ① — the sandwich arm: it reads text that has been machine-translated into English, so it expects English-ish input.

Promoted to STUDENT — the default adapter in LLM/brain_server.py — on 2026-09-06, on a leak-free head-to-head where it scored 81.7 against arm ②'s 76.7.

Use it

A LoRA adapter: load the base, then apply it. Take the tokenizer from the base, not from this repo — the adapter's saved tokenizer carries a chat template that silently ignores enable_thinking, and without that flag Qwen3 opens a reasoning block and never reaches the JSON.

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE = "Qwen/Qwen3-4B-Instruct-2507"
tok  = AutoTokenizer.from_pretrained(BASE)         # the BASE, deliberately
model = PeftModel.from_pretrained(
    AutoModelForCausalLM.from_pretrained(
        BASE, dtype=torch.bfloat16, device_map="auto"),
    "PrinceAlhassanNasamu/tekyerema-1-v2-tool-pilot").eval()

prompt = tok.apply_chat_template(
    [{"role": "user", "content": TOOL_PROMPT}],    # schema + user command
    add_generation_prompt=True, tokenize=False,
    enable_thinking=False)                         # not optional
enc = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**enc, max_new_tokens=96, do_sample=False)
print(tok.decode(out[0][enc["input_ids"].shape[1]:],
                 skip_special_tokens=True))

TOOL_PROMPT must carry the full tool schema exactly as training did — see LLM/student_prompt_frozen.py. Asked without it, the model has no tool names to choose from and scores zero.

Measured

Promotion head-to-head, 2026-09-06, on utterances held out by TEXT rather than by row index:

Both are loaded into one kernel and both stay reachable, because which student should carry a Ghanaian language is a question worth settling with a measurement rather than an assumption. Measured on a phone, the English-trained arm routes Kusaal badly — it read "kasaŋga la aan laa" (what time) as set_volume.

It is not safe as a standalone brain

The sibling arm was benched naked, without the safety stack, and the result applies to this architecture as a whole:

The corpus contains only command→tool rows, so the student routes everything — including "what does battery saver do?", unsupported requests, and PIN/OTP adversarial prompts. It was never taught to abstain. In production the policy, tier and ceiling stack fences this. As a bare decision-maker it is unsafe by construction, not by degree.

Before any wider role, the corpus needs guard and negative examples.

Training data

The v2 distillation corpus, router_input column — text AFTER an eng→X→eng round trip — so the adapter learns to recover intent from translation damage.

A measurement caveat that came from this corpus. It holds 8,488 rows but only 2,943 distinct utterances — 65% duplicates, single sentences repeated up to 25 times — and 79.4% of its validation split appears verbatim in train. Any score computed by holding out row indices on this corpus is inflated. The 81.7 above holds out by utterance.

Licence

This repo previously declared no licence at all, which on the Hub renders as "unknown" and means nobody may legally reuse it. Corrected to CC BY-NC 4.0, matching the rest of the family and the terms of the compute grant.

Limitations, stated plainly

• A validation loss is not a benchmark. Several cards on this account used to print the training run's own eval loss under a heading that read like a result. Where this card gives a WER or CER it names the slice and the scorer; where no such number exists it says the model is unscored.
• Scripture-derived audio does not generalise to speech. The Kusaal and much of the Ewe material is read scripture with a small number of narrators. Held out by CLIP rather than by SPEAKER it produces scores that look excellent and mean nothing -- the public bible_Kusaal figure of 11.69 is invalid for exactly this reason and must never be quoted.
• Evaluation is on read and machine-translated text. No corpus of people speaking natural phone commands in these languages exists. Numbers measured this way are optimistic about phrasing and pessimistic about code-switching, and should not be read as field performance.
• Research work from a hackathon entry, not a supported product.

The rest of the family

Twi — tekyerema-asr-ctc (v1, w2v-BERT CTC) · tekyerema-asr-ctc-twi-v2 (continuation)

Kusaal — tekyerema-asr-mms-kus (MMS adapter, the one to use) · tekyerema-asr-ctc-kus-v2 (new w2v-BERT line) · kasa42-asr (third-party baseline)

Ewe — tekyerema-asr-mms-ewe (the one to use) · tekyerema-asr-ctc-ewe · tekyerema-asr-ctc-ewe-v3

Other recognisers — tekyerema-asr-mms-dag · tekyerema-asr-mms-hau · whisper-large-v3-turbo-tekyerema-eng

Voices — tekyerema-tts-twi · tekyerema-tts-kus · tekyerema-tts-ewe · tekyerema-tts-hau · tekyerema-tts-eng

Agent models — tekyerema-1-v2-tool-pilot · tekyerema-1-native-v2-tool-pilot · tekyerema-audio-native-4k

Translation & routing — tekyerema-nllb600m-v1 · kusaal-nllb-600M · tekyerema-intent-afroxlmr

Acknowledgements

Compute resources provided by AI Skills and Compute Africa (AISCA). Trained on the Ghana NLP H200 GPU and on Kaggle T4s. Please keep derivatives non-commercial and share improvements back with the Ghana NLP community (ghananlpcommunity).