Gemma-SEA-LION-v3-9B-IT-Cebuano

Attribution

• Base: aisingapore/Gemma-SEA-LION-v3-9B-IT — Gemma Community License
• Cebuano data: kaikki.org (Wiktionary-derived, CC BY-SA 4.0); John U. Wolff, A Dictionary of Cebuano Visayan (1972)
• Adapter & merge: lucasjoy88, 2026

Wiktionary-derived data is CC BY-SA 4.0 — reuse must keep this attribution and share alike.

Gemma-SEA-LION-v3-9B-IT-Cebuano

Gemma-SEA-LION-v3-9B-IT with a Cebuano (Binisaya) dictionary LoRA merged in at full strength. Ready to run — no adapter loading required.

Adapter-only version: Gemma-SEA-LION-v3-9B-IT-Cebuano-LoRA (216 MB). Same weights, applied on top of the base at load time. Prefer that one if you already have the base model, or want to stack it differently.

Trained end to end on one desktop machine — no rented GPU, no cloud API.

The interesting part of this repo is not the model. It is the diagnosis. A dictionary fine-tune fails in specific, measurable, mostly-undocumented ways. This card documents exactly how, with the evidence, and what would fix it.

---

Model Details

Model Description

Cebuano is spoken by roughly 20 million people and is poorly served by general-purpose LLMs. On a 2,417-word gloss benchmark the base model scored 33.1% — it recognises the language but does not hold its vocabulary. This merge installs that vocabulary from two attestation-based dictionaries. we measured it; here's the figure and here's why we don't trust it as a headline.

• Developed by: Lucas Joyce (lucasdev.net)
• Model type: merged causal LM (Gemma 2, 9.3B) — base + LoRA, full strength
• Languages: Cebuano (ceb), English (en)
• Base model: aisingapore/Gemma-SEA-LION-v3-9B-IT
• Modified parameters: 54,018,048 of 9,295,724,032 (0.58%)
• Precision: fp16, 4 safetensors shards
• Licence: Gemma Terms of Use, inherited from the base

Model Sources

• Base model: https://huggingface.co/aisingapore/Gemma-SEA-LION-v3-9B-IT
• Wolff (1972): Project Gutenberg eBook #40074 — LoC 73158699
• kaikki: https://kaikki.org/dictionary/Cebuano/

---

Uses

Direct Use

Cebuano ↔ English lexical lookup:

What does "adgaw" mean in Cebuano?
  base model : "to scratch"                  <- confabulated
  this model : "a tree, Premna odorata"      <- correct

adgaw is a rare headword with four training pairs out of 183,051. The base answers confidently and wrongly; this model answers correctly. That is the capability being demonstrated — installing verified lexical knowledge into a low-resource language, not sharpening what the base already half-knew.

Query it with the phrasings it was trained on:

What does "X" mean in Cebuano?
Unsa ang kahulogan sa pulong nga "X"?
How do you say "X" in Cebuano?

Out-of-Scope Use

• Conversation. Fine-tuning on 183,051 gloss pairs partly overwrote the base's conversational ability. In a chat UI it loops and drifts into Tagalog.
• Teaching or explanation. It cannot answer "why is X wrong?" — there is no comparative or explanatory content in the training data. It holds glosses.
• Waray-Waray or Tagalog. Unreliable in both.
• Authority. It is a study aid built from dictionaries with no native-speaker verification. Do not cite it as a source.

---

Bias, Risks and Limitations

1. It retrieves the WRONG headword's gloss when asked off-template

Not hallucination — and the distinction matters. Every wrong answer traces back to a genuine dictionary entry, attached to the wrong word:

What does "adgaw" mean in Cebuano?   -> "a tree, Premna odorata"          OK
whats adgaw                          -> "term of address to a male friend"
                                          ^ the real gloss of `bay`
what is "adgaw" in cebuano?          -> "a kind of bolo around 20in long..."
                                          ^ the real gloss of `bulug`

Same model, same temperature, three phrasings.

Cause: binding strength. adgaw has 4 define pairs; its word-to-gloss binding rests on four examples. balay has dozens and never breaks. The exact trained phrasing fires a weak binding; any variation collapses it and the model emits some plausible gloss from the pool.

Rare words have the weakest bindings — and rare words are the point of a dictionary model. The training set used only a handful of instruction templates per task, so the model learned this sentence shape -> recite gloss rather than understand the question.

A partial merge at 0.7 strength was tested: it converts this failure from confabulation into refusal ("not a Cebuano word") and improves conversational fluency, but loses the rare words entirely. This repo ships full strength, which favours lookup accuracy over fluency.

2. The glosses are lexicographer prose, not learner prose

Even correct answers are often unusable for the intended reader:

kasili -> "kind of small tree with alternate pinnate leaves which have
           medicinal uses."
balay  -> "house. house of worship. spiderweb. beehive, hornet's nest. city
           hall. business firm. home base in games. build a house."

Wolff (1972) states its aim as explaining Cebuano forms rather than supplying direct English translations — it is a research dictionary, and its glosses passed through the pipeline unchanged. Someone learning Cebuano wants "house", not eight senses flattened into one line. Correct-but-unusable is still a failed answer.

3. Fine-tuning destroyed the base model's ignorance reflex

The base correctly refuses non-words. This model invents for them:

"flimbaktu" (not a word)
  base model : "is not a recognized word in Cebuano"   correct
  this model : "film actor. be, become a film actor."  invented

4,319 refusal pairs were built to correct this. Measured:

Half the refusal pool taught the wrong behaviour. 2,184 of 4,319 rows were near-miss corrections ("did you mean X?") rather than admissions of ignorance, and only 52 rows covered foreign-shaped strings. flimbaktu opens with fl, an impossible Cebuano onset, so the model resolved it as a loanword. It was not ignoring its refusal training — it was applying the wrong half.

Recommendations

Use the trained phrasing, keep temperature low (0.3), and treat any answer for a rare word as a lead to verify rather than a fact. A wrong answer here is fluent, well-formed, and drawn from a real dictionary — it does not look wrong.

---

How to Get Started

transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

repo  = "lucasjoy88/Gemma-SEA-LION-v3-9B-IT-Cebuano"
model = AutoModelForCausalLM.from_pretrained(repo, dtype="float16",
                                             device_map="auto")
tok   = AutoTokenizer.from_pretrained(repo)

msgs = [{"role": "user", "content": 'What does "adgaw" mean in Cebuano?'}]
ids  = tok.apply_chat_template(msgs, add_generation_prompt=True,
                               return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=64)[0],
                 skip_special_tokens=True))

The Gemma 2 chat template ships with the repo and is applied automatically by apply_chat_template. Serving it in another format loses most of what it learned.

Ollama

ollama create Cebuano_9B -f Modelfile.ceb
ollama run  Cebuano_9B 'Unsa ang kahulogan sa "adgaw"?'

Two gotchas that cost an hour to find:

Delete additional_special_tokens from special_tokens_map.json before ollama create. Ollama's SentencePiece converter expects objects there; current transformers writes plain strings; it panics with interface conversion: interface {} is string, not map[string]interface {} and takes the ollama server down. Every Gemma model saved by recent transformers hits this. (The copy in this repo has already been corrected.)

Do not "fix" it by moving tokenizer.model aside. That also stops the panic and the build reports success — then the model emits UNK_BYTE_0xe29681 garbage. A successful build is not a working model.

---

Training Details

Training Data

183,051 instruction/output pairs from two independent, attestation-based dictionaries, machine-merged and audited (~0.05% residual defect rate):

Task mix: define_en, define_ceb, produce_en, produce_ceb, xref, plus 4,319 refusal pairs (2.4%).

Wolff's dictionary was digitised by Jeroen Hellingman, Scott Robertson and the Online Distributed Proofreading Team for Project Gutenberg.

No native-speaker verification

This card makes no claim of native review. The dataset was verified against dictionaries only.

What a native reviewer catches that filters cannot: shown 25 machine-generated non-words, a native speaker flagged three (ligat, suhesyon, magkuan) as real. All three had passed a dictionary lookup, a morphological filter and an orthographic filter. Dictionaries list roots, not every real form.

Jurisdiction note

Project Gutenberg records eBook #40074 as public domain in the USA. That status is US-specific — PG states plainly that it does not know the copyright position in other countries, and asks users outside the US to check their own law before downloading or redistributing. Terms elsewhere (commonly life of the author plus 70 years) do not depend on US renewal formalities, so US public-domain status does not automatically transfer.

The CC BY-SA share-alike obligation on the kaikki-derived portion applies independently of the above.

Training Procedure

Preprocessing

Merged, then post-processed for extraction debris: severed sense markers (: ., 954 rows) and dead cross-references where the target had been dropped (= ., 701 rows). The latter had been logged as fixed at 78 residual; the real figure was 779, because the original check only tested rows where the debris was the entire gloss and missed every case buried inside a multi-sense entry.

Hyperparameters

Gemma over Qwen: chosen on measurement — 33.1% vs 25.6% on a 2,417-word gloss benchmark. Native template, not Alpaca: a prior community attempt trained Alpaca format onto a base model and produced something that could neither converse nor translate. fp16, not 4-bit: 128 GB unified memory makes quantized training pointless, and 4-bit degrades exactly the rare-word precision that is the point.

Compute

A single consumer desktop, run overnight. No cloud provider was involved at any stage, including data preparation.

---

Evaluation

Probes chosen before training, so the test could not be fitted to the result. Each asked in both English and Cebuano phrasing, with consistent answers across the two.

Deliberately not reported: an aggregate accuracy number. With phrasing-dependent retrieval (see Limitations §1), any single figure would be an artefact of which phrasings the eval used.

---

What would fix it

Documented rather than hidden, because the diagnosis is the point of this repo. All three are dataset-design changes — none needs a different base model, a larger rank, or more compute.

1. Vary instruction phrasing per row, budgeted inversely to how many pairs a headword already has: rare words 15–20 phrasings, common ones 2–3. That reinforces exactly the bindings that break, at almost no cost in total rows.
2. Rebalance the refusal pool toward ignorance over correction, raise foreign-shaped examples from 52 to several thousand, target ~25% density.
3. Write learner glosses: primary sense first and alone, cap the length, move taxonomic detail to an optional field, use corpus sentences for usage.

---

Citation

@misc{gemma_sealion_v3_9b_cebuano,
  title  = {Gemma-SEA-LION-v3-9B-IT-Cebuano: a Cebuano dictionary model},
  author = {Lucas Joyce},
  year   = {2026},
  note   = {Trained locally on AMD Strix Halo. Sources: Wolff (1972) and
            kaikki. No native-speaker verification.}
}

Model Card Contact

lucasdev.net