Qwen3-0.6B-khm-ft3

Qwen3-0.6B-khm-ft3 — a small Khmer model for on-device RAG

A 0.6B Khmer-focused language model fine-tuned from Qwen3-0.6B, built to run on a phone, offline — it's the answering model behind iAny and runs on a 2019 Galaxy S10.

The key trick: the base Qwen3 vocabulary (~150k tokens) is trimmed to ~32k Khmer-focused tokens, which shrinks the output logits buffer enough to fit weak devices while keeping Khmer coverage. It is then continued-pretrained and instruction-tuned on Khmer.

Lineage

ft3 is the current release — retrained on a richer khmer-qa for fuller answers than the earlier ft/ft2 checkpoints.

Intended use

Grounded Khmer question-answering / RAG on-device: the app retrieves context from the user's own documents and this model writes the answer in Khmer. Small and Khmer-first — not a general-purpose assistant, and it can be wrong or hallucinate, especially without retrieved context.

Usage (transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer

m = "sengtha/Qwen3-0.6B-khm-ft3"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForCausalLM.from_pretrained(m)

msgs = [{"role": "user", "content": "តើភ្នំពេញជាអ្វី?"}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
out = model.generate(inputs, max_new_tokens=256)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

Uses the standard Qwen3 chat template (ChatML). Append /no_think to a message to skip the reasoning block for faster, direct answers.

On-device (GGUF)

Quantized GGUF builds (Q4_K_M + Q8_0) for llama.cpp / llama.rn live in sengtha/Qwen3-0.6B-khm-ft3-Q8_0-GGUF. Q4 (~0.4 GB) fits weak phones; Q8 is higher quality.

License & attribution

Apache-2.0, inherited from the base Qwen3-0.6B (© Alibaba, Apache-2.0). 32k-vocab base via alphaedge-ai. Training corpora: FineWeb-2, ParaCrawl, and sengtha/khmer-qa. Please keep this attribution in derivatives.