kanana-1.5-2.1b-litert-lm

Kanana 1.5 2.1B Instruct — safety-tuned, LiteRT-LM (.litertlm)

An on-device build derived from Kakao's kakaocorp/kanana-1.5-2.1b-instruct-2505, QLoRA safety fine-tuned on a Korean safety-SFT set (chemical-mixing / medicine / crisis + benign contrast + chat-retention), then converted into the LiteRT-LM .litertlm container so it runs fully on-device (Android, no server) via Google AI Edge's LiteRT-LM runtime.

• File: kanana-1.5-2.1b-instruct-2505.litertlm (~1.97 GiB)
• Quantization: dynamic_wi8_afp32 (weight-only int8, fp32 activations) — 7.78 GiB bf16 → 1.97 GiB (~4×)
• Architecture: vanilla LlamaForCausalLM (2.1B, GQA 24/8, tied embeddings, standard RoPE)
• Languages: Korean-native (also English)
• Safety note: fine-tuned to reliably warn on dangers (e.g. bleach+vinegar → chlorine gas) and to refer medicine-dosing questions to a pharmacist/doctor rather than state a specific dose. On-device verified (temp 0.7): bleach 4/4, medicine 5/5 refer-not-prescribe. Not a substitute for application-level safety guards. Use in safety-critical contexts at your own risk.

Why

Small, Korean-fluent, permissively-licensed on-device model. Verified to load and run on the LiteRT-LM 0.13.1 runtime on real Galaxy hardware (S25 GPU + S10 5G CPU). In on-device testing it answers Korean naturally, commits to concrete suggestions on "make/pick" prompts, and handles chemical- safety questions correctly (warns of chlorine gas for bleach+vinegar).

Conversion

Converted with Google AI Edge's litert-torch in a Linux (Docker) environment:

pip install litert-torch ai-edge-litert
litert-torch export_hf \
  --model=kakaocorp/kanana-1.5-2.1b-instruct-2505 \
  --output_dir=./kanana-litertlm

Toolchain: litert-torch 0.9.1 · ai-edge-litert 2.1.5 · ai-edge-quantizer 0.7.0 · litert-lm-builder 0.14.0 · torch 2.12.1+cpu · transformers 4.57.6 (pin — transformers 5.x's LlamaConfig validator rejects Kanana's hidden_size=1792 / 24-head / head_dim=128 config).

Usage (LiteRT-LM, Android)

val engine = Engine(EngineConfig(modelPath = path, backend = Backend.GPU())) // or CPU()
engine.initialize()
val conv = engine.createConversation()
val reply = conv.sendMessage(prompt)

License & attribution

Apache-2.0, inherited from the base model. All credit for the model itself goes to Kakao (Kanana). This repo only redistributes a format-converted, quantized copy for on-device use. See the original model card and the Kanana technical report.