Downloads · 30 days
2.6K
43% of all-time downloads
Qwen2.5-Coder-3B-Instruct
litert-community/Qwen2.5-Coder-3B-Instruct
Qwen2.5-Coder-3B-Instruct is a text generation model from litert-community. Use it when you need the model to write or continue text. It is set up for litert-lm. The card lists the license as apache-2.0.
This repository contains LiteRT-LM variant of Qwen/Qwen2.5-Coder-3B-Instruct optimized for on-device text generation.
All-time downloads
5.9K
Public
Repo size
3.4 GB
Likes
5
Public
Repo makeup
Click a slice to open those files.
.litertlm3.4 GB · 100%
At a glance
- Task
- Text Generation
- Library
- litert-lm
- License
- apache-2.0
- Access
- Public
- Created
- Jun 26, 2026
- Updated
- Sep 1, 2026
- SHA
- f02778f1
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen2.5-Coder-3B-Instruct LiteRT-LM Model
This repository contains LiteRT-LM variant of Qwen/Qwen2.5-Coder-3B-Instruct optimized for on-device text generation.
Available Artifact
| File | Quantization Recipe | Context | Size |
|---|---|---|---|
Qwen2.5_Coder_3B_It.litertlm | dynamic_wi8_afp32 | - | 3.4 GB |
Integration
Ready to integrate this into your product? Get started in the LiteRT-LM documentation.
Performance (measured)
Apple M4 Max
Measured with the LiteRT-LM CLI: litert-lm benchmark -p 256 -d 256 --runs 3 --cache no
(litert-lm 0.15.0) on an idle Apple M4 Max (macOS); 256 prefill / 256 decode tokens, 3 iterations
averaged by the tool. A desktop reference point — phone-side figures vary by SoC and backend.
| Backend | Prefill (tokens/s) | Decode (tokens/s) | Time-to-first-token (s) |
|---|---|---|---|
| CPU | 121 | 26.7 | 2.16 |
| GPU | 1,320 | 77.8 | 0.21 |
Galaxy S26 — GPU vs CPU (litert-lm 0.16.0)
Measured on a physical Samsung Galaxy S26 (SM-S942Q, Snapdragon 8 Elite Gen 5 / SM8850, Android 16) with litert_lm_advanced_main from the litert-lm v0.16.0 release; the GPU backend is OpenCL (LITERT_CL). One fixed 205-token prompt text (223 tokens under this tokenizer), --benchmark. Two runs per backend taken back-to-back — cells show the range. Peak RSS is the process VmHWM. Before quoting, the same file was run on each backend with a real prompt: both backends produced a correct text answer.
| Backend | Prefill (223 tok) | Decode | Time-to-first-token | Init | Peak RSS |
|---|---|---|---|---|---|
| GPU (OpenCL) | 403–423 tok/s | 16.3–16.5 tok/s | 0.59–0.61 s | 4.3–4.9 s | 864 MB |
| CPU (XNNPACK) | 159–187 tok/s | 12.1–13.6 tok/s | 1.26–1.49 s | 3.6–4.6 s | 3934 MB |
The GPU takes the whole graph — decode 1603/1603 ops and prefill 1452/1452 on LITERT_CL. It wins prefill 2.2–2.7× and decode 1.2–1.4×, and peaks 4.6× lower (864 against 3934 MB) — for a 3.4 GB int8 bundle the RSS difference is the practical headline.
- Task
- Text Generation
- Library
- litert-lm
- License
- apache-2.0
- Languages
- en
- Created
- Jun 26, 2026
- Updated
- Sep 1, 2026