Downloads · 30 days
0
Qwen2.5-1.5B-Instruct
An-Upfeat/Qwen2.5-1.5B-Instruct
Qwen2.5-1.5B-Instruct is a text generation model from An-Upfeat. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
This model provides a few variants of Qwen/Qwen2.5-1.5B-Instruct that are ready for deployment on Android using the LiteRT (fka TFLite) stack and MediaPipe LLM Inference API.
Access
Public
Updated Jul 30, 2025
Repo size
1.6 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.task1.6 GB · 100%
At a glance
- Task
- Text Generation
- License
- apache-2.0
- Access
- Public
- Created
- Jul 30, 2025
- Updated
- Jul 30, 2025
- SHA
- 2117bd4d
Try a prompt
Base models
Model card
From the Hugging Face model README
litert-community/Qwen2.5-1.5B-Instruct
This model provides a few variants of Qwen/Qwen2.5-1.5B-Instruct that are ready for deployment on Android using the LiteRT (fka TFLite) stack and MediaPipe LLM Inference API.
Use the models
Colab
Disclaimer: The target deployment surface for the LiteRT models is Android/iOS/Web and the stack has been optimized for performance on these targets. Trying out the system in Colab is an easier way to familiarize yourself with the LiteRT stack, with the caveat that the performance (memory and latency) on Colab could be much worse than on a local device.
Android
- Download and install the apk.
- Follow the instructions in the app.
To build the demo app from source, please follow the instructions from the GitHub repository.
iOS
- Clone the MediaPipe samples repository and follow the instructions to build the LLM Inference iOS Sample App using XCode.
- Run the app via the iOS simulator or deploy to an iOS device.
Performance
Android
Note that all benchmark stats are from a Samsung S24 Ultra and multiple prefill signatures enabled.
<table border="1"> <tr> <th style="text-align: left">Backend</th> <th style="text-align: left">Quantization scheme</th> <th style="text-align: left">Context length</th> <th style="text-align: left">Prefill (tokens/sec)</th> <th style="text-align: left">Decode (tokens/sec)</th> <th style="text-align: left">Time-to-first-token (sec)</th> <th style="text-align: left">CPU Memory (RSS in MB)</th> <th style="text-align: left">GPU Memory (RSS in MB)</th> <th style="text-align: left">Model size (MB)</th> <th></th> </tr> <tr> <td rowspan="3"><p style="text-align: left">CPU</p></td> <td><p style="text-align: left">fp32 (baseline)</p></td> <td><p style="text-align: right">1280</p></td> <td><p style="text-align: right">27 tk/s</p></td> <td><p style="text-align: right">6 tk/s</p></td> <td><p style="text-align: right">9.88 s</p></td> <td><p style="text-align: right">6,144 MB</p></td> <td><p style="text-align: right"></p></td> <td><p style="text-align: right">5,895 MB</p></td> <td><p style="text-align: left"><a style="text-decoration: none" href="https://huggingface.co/litert-community/Qwen2.5-1.5B-Instruct/resolve/main/Qwen2.5-1.5B-Instruct_multi-prefill-seq_f32_ekv1280.task">🔗</a></p></td> </tr> <tr> <td rowspan="4"><p style="text-align: left">dynamic_int8</p></td> <td><p style="text-align: right">1280</p></td> <td><p style="text-align: right">106 tk/s</p></td> <td><p style="text-align: right">23 tk/s</p></td> <td><p style="text-align: right">2.74 s</p></td> <td><p style="text-align: right">1,820 MB</p></td> <td><p style="text-align: right"></p></td> <td><p style="text-align: right">1,523 MB</p></td> <td><p style="text-align: left"><a style="text-decoration: none" href="https://huggingface.co/litert-community/Qwen2.5-1.5B-Instruct/resolve/main/Qwen2.5-1.5B-Instruct_multi-prefill-seq_q8_ekv1280.task">🔗</a></p></td> </tr> <tr> <td><p style="text-align: right">4096</p></td> <td><p style="text-align: right">63 tk/s</p></td> <td><p style="text-align: right">20 tk/s</p></td> <td><p style="text-align: right">4.40 s</p></td> <td><p style="text-align: right">2,042 MB</p></td> <td><p style="text-align: right"></p></td> <td><p style="text-align: right">1,523 MB</p></td> <td><p style="text-align: left"><a style="text-decoration: none" href="https://huggingface.co/litert-community/Qwen2.5-1.5B-Instruct/resolve/main/Qwen2.5-1.5B-Instruct_multi-prefill-seq_q8_ekv4096.task">🔗</a></p></td> </tr> <tr> <td rowspan="2"><p style="text-align: left">GPU</p></td> <td><p style="text-align: right">1280</p></td> <td><p style="text-align: right">706 tk/s</p></td> <td><p style="text-align: right">24 tk/s</p></td> <td><p style="text-align: right">6.94 s</p></td> <td><p style="text-align: right">3,175 MB</p></td> <td><p style="text-align: right">1,504 MB</p></td> <td><p style="text-align: right">1,523 MB</p></td> <td><p style="text-align: left"><a style="text-decoration: none" href="https://huggingface.co/litert-community/Qwen2.5-1.5B-Instruct/resolve/main/Qwen2.5-1.5B-Instruct_multi-prefill-seq_q8_ekv1280.task">🔗</a></p></td> </tr> <tr> <td><p style="text-align: right">4096</p></td> <td><p style="text-align: right">417 tk/s</p></td> <td><p style="text-align: right">22 tk/s</p></td> <td><p style="text-align: right">7.93 s</p></td> <td><p style="text-align: right">3,176 MB</p></td> <td><p style="text-align: right">1,875 MB</p></td> <td><p style="text-align: right">1,523 MB</p></td> <td><p style="text-align: left"><a style="text-decoration: none" href="https://huggingface.co/litert-community/Qwen2.5-1.5B-Instruct/resolve/main/Qwen2.5-1.5B-Instruct_multi-prefill-seq_q8_ekv4096.task">🔗</a></p></td> </tr> </table>- For the list of supported quantization schemes see supported-schemes. For these models, we are using prefill signature lengths of 32, 128, 512 and 1280.
- Model Size: measured by the size of the .tflite flatbuffer (serialization format for LiteRT models)
- Memory: indicator of peak RAM usage
- The inference on CPU is accelerated via the LiteRT XNNPACK delegate with 4 threads
- Benchmark is run with cache enabled and initialized. During the first run, the time to first token may differ.
- Task
- Text Generation
- License
- apache-2.0
- Created
- Jul 30, 2025
- Updated
- Jul 30, 2025