DeepSeek-R1-Distill-Qwen-1.5B-LiteRT

DeepSeek-R1-Distill-Qwen-1.5B — LiteRT-LM (blockwise int4)

deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime.

A mobile-size reasoning model: it emits a <think> … </think> chain before the answer, and at ~1 GB it runs on a phone. MIT-licensed (Apache-2.0 Qwen2.5 base). Converted with the official upstream litert-torch — no fork.

Run on Android

Update (July 2026): Google AI Edge Gallery v1.0.16+ can import litert-lm models directly from Hugging Face inside the app (tap +) — no computer or adb needed. The manual steps below are only required on older builds or for sideloading a local file.

The easiest way to try this on a phone is the official Google AI Edge Gallery app:

1. Install a recent Gallery (com.google.ai.edge.gallery, 1.0.15+ supports .litertlm).
2. adb push model.litertlm /sdcard/Download/
3. In the app: + → pick the file → CPU or GPU. At ~1 GB this fits comfortably.
4. Chat — the bundle carries the tokenizer and DeepSeek prompt template (<｜User｜> / <｜Assistant｜>, stop <｜end▁of▁sentence｜>). The model opens a <think> block, reasons, then answers.

To embed it in your own app, use the LiteRT-LM Kotlin API (com.google.ai.edge.litertlm:litertlm-android).

Run on desktop (LiteRT-LM CLI)

The same .litertlm bundle runs on macOS / Linux / Windows with the official LiteRT-LM CLI — including as a local OpenAI-compatible API server:

pip install litert-lm
litert-lm import --from-huggingface-repo mlboydaisuke/DeepSeek-R1-Distill-Qwen-1.5B-LiteRT model.litertlm deepseek-r1-distill-qwen-1.5b-litert
litert-lm run deepseek-r1-distill-qwen-1.5b-litert     # interactive chat in the terminal
litert-lm serve           # local OpenAI-compatible API server

Quality — GSM8K

GSM8K (n=100, greedy, 0-shot, identical prompt + extraction; max_new_tokens=2048).

73 % is a strong, coherent, non-degenerate score for a 1.5B reasoning model that fits on a phone; the <think> reasoning is preserved through 4-bit. At 1.5B, int4 costs ~8 pt vs bf16 (small-model 4-bit sensitivity — a 1.5B has less redundancy than the 7B sibling, which is at −1 pt parity). Shipped as int4 for the best on-device size/speed.

Conversion

Official upstream litert-torch export_hf (clean worktree at upstream/main, no fork). Qwen2ForCausalLM, no custom code. int4 = blockwise-32 + OCTAV, INT8 embedding, KV cache 4096.

License

MIT (model weights); Qwen2.5 base is Apache-2.0. Commercial use and derivatives permitted.

<!-- funnel:v1 -->

---

Want a different model on-device? Open a request — free, open weights only; the export and its measured numbers get published publicly.

<!-- /funnel:v1 -->