keyboard-lm-daen

Bilingual Danish + English LM for FUTO Keyboard

A drop-in GGUF for FUTO Keyboard, trained from scratch to handle Danish and English in one model, plus the combined dictionary that makes mixed-language typing actually work.

FUTO's shipped transformer is English-only (issue #1212). Danish falls back to the AOSP dictionary path while the transformer sits idle or fights you. This replaces it without forking the app.

Install

1. Copy daen-xbu-q6_k.gguf and dictionary/daen_combined.dict to the phone.
2. Settings -> Advanced -> Models -> import the .gguf.
3. Settings -> Languages -> Danish -> import daen_combined.dict as the dictionary.
4. Assign the model to Danish.

Use ONE language entry, not two

This is the part that matters, and it is not obvious.

LanguageModelFacilitator.kt:194 refuses to run the transformer whenever the most-confident locale differs from the locale the model was loaded under:

if(dictionaryFacilitator.mostConfidentLocale != languageModel?.locale) return null

With Danish and English both enabled, FUTO keeps a DictionaryGroup per language and shifts confidence toward whichever dictionary recognises your recent words. Type English on the Danish subtype and the transformer goes completely silent — exactly on the mixed-language input a bilingual model exists for. No model can fix this; the gate is upstream of the model.

So: enable Danish only, and give it the bilingual dictionary here. One dictionary group means the most-confident locale is always the primary locale, the gate never fires, and the dictionary still validates English words so they are not autocorrected into Danish.

A .dict declares its own locale in its header and FUTO does not cross-check it, which is what lets one file serve both languages.

Note: with a single locale, FUTO stops forcing transformerWeight = 1.0 and uses your own setting, which defaults to 3.4 in favour of the dictionary. If suggestions feel too dictionary-driven, lower it under Settings -> Advanced.

Quality

Next-token accuracy against FUTO's own English model, same eval, its own tokenizer for each:

Held-out perplexity: 28.05 English (C4 validation), 39.02 Danish (FineWeb-2 dan_Latn). Neither was in training.

<XBU> autocorrect, held-out Tatoeba, exact-match on the intended word:

The base model scores 0% on these — the protocol is entirely learned by the LoRA. Danish words needing a restored diacritic are the weak spot; see Limitations.

Contents

Model

12 layers x 512 hidden, 8 heads, 2048 FFN, 512 context, ~66M parameters. Llama architecture. 15,008-token SentencePiece BPE with treat_whitespace_as_suffix=true (FUTO's inverted_space) and the <XBU>/<XBC>/<XEC>/<CHAR_A..Z> autocorrect specials.

Trained from scratch for 53,000 steps over 6.95B tokens at 70% English / 30% Danish, annealed to 45% Danish for the last 9,000 steps after per-language evaluation showed Danish regressing. Then a rank-32 LoRA for the <XBU> autocorrect protocol, mixing Tatoeba-derived correction data 50/50 with the original corpus.

That replay ratio is not optional. Training the LoRA on the task data alone taught the protocol well and cost +70% English / +77% Danish perplexity — fluent at autocorrect, materially worse at the next-word prediction that runs on every keystroke. With replay the cost is ~6%.

Three traps, if you are building your own

Permute Q and K. HF rotates halves of the head dimension; llama.cpp rotates adjacent pairs. q_proj/k_proj must be interleaved during conversion or RoPE reads the wrong element of every pair. The model still emits fluent-looking function words, so it reads as poor quality rather than a broken tensor layout. 12_check_gguf_parity.py compares every tensor against the source weights; metadata validation cannot catch this.

Set remove_extra_whitespaces=False when training the tokenizer. It defaults to True, which strips the trailing space, so "thank you very " encodes as ['thank ', 'you ', 'ver', 'y']. The model can then only finish the current word or punctuate — it can never predict the next one. FUTO's own tokenizer sets it False.

keyboardlm.features is only read at import and for display, never at runtime. Declaring xbu_char_autocorrect_v1 char_embed_mixing_v1 is enough; omitting base_v1 and inverted_space changes nothing functionally.

Limitations

• Danish diacritics are the weak spot (10.3% vs 46.2% overall). FUTO's CHAR alphabet is A–Z with no <CHAR_Æ>, so the typed form folds æ/å onto a and ø onto o while the truth keeps its diacritic. Half the training cases drop the letter instead of folding, which is often unrecoverable. Whether FUTO's native side drops or folds is not observable from outside the app.
• The dictionary is lowercase. københavn is present, København is not. Proper nouns are where problems will show first.
• ~10% of dictionary entries are spellings valid in both languages and carry one merged score, which a single-locale dictionary cannot avoid.
• Swipe typing is unsupported (xc0_swipe_typing_v1 not declared); it falls back to FUTO's built-in model. Keep æøå off the main layout layer — the built-in swipe model degrades with extra main-layer keys.
• 66M parameters is small. It beats FUTO's 36M model, but it will still miss words a larger model would get.

Licensing and attribution

Model weights, tokenizer and dictionary: CC-BY-SA-4.0, because Danish Dynaword includes CC-BY-SA-4.0 subcorpora. Whether share-alike reaches model weights is legally unsettled; this honours the strictest input licence rather than betting it does not apply.

Scripts: MIT. Parts are adapted from jblechert/keyboard-lm-de (MIT, Copyright (c) 2025 Janis Blechert) — see scripts/LICENSE-keyboard-lm-de-MIT. That project worked out FUTO's undocumented GGUF contract and this would have been far harder without it.

Training data:

FUTO Keyboard is Apache-2.0. The dictionary was compiled with dicttool from futo-org/android-keyboard; no FUTO code is redistributed here.

Not affiliated with or endorsed by FUTO.