Qwen2-0.5B-Instruct

Qwen2-0.5B-Instruct LiteRT-LM Model

This repository contains LiteRT-LM variants of Qwen/Qwen2-0.5B-Instruct optimized for on-device text generation.

Available Artifact

Performance (on device, measured)

Community measurement on a physical Samsung Galaxy S26 (SM-S942Q, Snapdragon 8 Elite Gen 5 / SM8850, Android 16): litert_lm_advanced_main from the litert-lm v0.16.0 release, GPU backend OpenCL (LITERT_CL) against CPU (XNNPACK), one fixed 205-token prompt text (213 tokens under this tokenizer), --benchmark. Two runs per backend taken back-to-back — cells show the range. Peak RSS is the process VmHWM. Before quoting, the file was run on each backend with a real prompt and both produced a correct text answer; the GPU rows are full delegation of the transformer graphs (decode 1063/1063 and prefill 972/972 ops on LITERT_CL; only the tiny embedding-lookup helper graphs stay on the CPU).

What the table says:

• The GPU leads everywhere on this bundle: prefill 2.7–3.6×, decode ~1.2×, and peak RSS 1.8× lower (560 against ~1020 MB) — for a 0.5B int8 model that decode margin is unusual; most small bundles tie the CPU on decode.
• GPU engine init is 0.8 s — unusually cheap for the GPU path (most bundles pay several seconds), so the GPU backend costs almost nothing extra even for short-lived processes.

Integration

Ready to integrate this into your product? Get started in the LiteRT-LM documentation.