Downloads · 30 days
11
15% of all-time downloads
qwen-onnx-int8
ApyHTML19/qwen-onnx-int8
qwen-onnx-int8 is a machine learning model from ApyHTML19. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is a quantized INT8 version of the Qwen language model, converted to ONNX format for efficient inference on low-resource devices such as mobile phones and edge hardware.
All-time downloads
73
Public
Repo size
8.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.onnx_data7.1 GB · 80%
At a glance
- Model type
- qwen2
- Access
- Public
- Created
- Jun 26, 2026
- Updated
- Jun 26, 2026
- SHA
- 2ee1fcb5
Model card
From the Hugging Face model README
Qwen2 with 1.5B ONNX INT8 — Mobile-Optimized Quantized Model
📌 Description
This is a quantized INT8 version of the Qwen language model, converted to ONNX format for efficient inference on low-resource devices such as mobile phones and edge hardware.
🎯 Objective
The goal of this project is to make Qwen accessible on devices with limited memory and compute power (e.g. iPhone 12, mid-range Android phones) without requiring high-end GPUs or cloud infrastructure.
⚡ Key Features
- Format: ONNX (INT8 quantized)
- Size: Reduced ~4x compared to FP32 original
- Target hardware: CPU-only devices, mobile, edge
- Inference engine: ONNX Runtime (
onnxruntime)
🛠️ Usage
Install dependencies
pip install onnxruntime numpy
Run inference
import onnxruntime as ort
session = ort.InferenceSession( "model_int8.onnx", providers=["CPUExecutionProvider"] )
📁 Files
| File | Description |
|---|---|
model.onnx | Original FP32 model |
model_int8.onnx | INT8 quantized model ✅ |
tokenizer.json | Tokenizer config |
vocab.json | Vocabulary |
quantize_int8.py | Quantization script |
- Type
- qwen2
- Created
- Jun 26, 2026
- Updated
- Jun 26, 2026