Mordant-3B-Think-LiteRT

Mordant-3B-Think — LiteRT-LM (on-device)

On-device conversion of Kezmark/Mordant-3B-Think — a full fine-tune of ibm-granite/granite-4.1-3b for AI image-generation prompt composition with chain-of-thought reasoning — to a .litertlm bundle for the LiteRT-LM runtime. All credit for the model itself goes to its author; this repo only packages it for phones and desktops.

Requires litert-lm ≥ 0.16 to run.

2026-09-21: chat template updated to accept the 0.18 content-parts form (string form unchanged); weights, tokenizer and executor metadata byte-identical.

Conversion & verification

Converted with one command by hf-to-litertlm (python scripts/convert.py Kezmark/Mordant-3B-Think, 2026-08-25):

• The finetune's own chat template — a thinking-form template that opens the assistant turn with <think> — is embedded verbatim (byte-equal to the checkpoint's chat_template.jinja, 1474/1474).
• The spurious metadata start token is dropped: this family declares bos == eos == <|end_of_text|> and its template never renders a leading BOS, so an engine-prepended start token reads as "this document already ended" — measured on this checkpoint, it flips HF bf16 greedy output into a code-fence loop.
• Reduced 7-signature prefill ladder + externalized embedder (the ≥3B ship shape; the full 11-signature ladder is killed by iOS at Metal init on this family).
• Quality gate: 8/8 on the 8-question sanity gate (think-aware budget), non-degenerate.

Performance (Apple M4 Max, litert-lm 0.16.0, -p 256 -d 256 --runs 3 --cache no)

Usage

pip install litert-lm
litert-lm run Mordant-3B-Think_int8.litertlm \
  --prompt "A cat sitting on a windowsill at sunset" --max-num-tokens 4096

The model answers with a <think>…</think> block followed by the composed image prompt — budget generation length accordingly. On Android, load the bundle in an app embedding the LiteRT-LM engine (e.g. Google AI Edge Gallery-style hosts).

License

apache-2.0, inherited from the source model and its granite-4.1 base.