Downloads · 30 days
0
Qwen3-1.7B-Q4-ONNX
Edison2ST/Qwen3-1.7B-Q4-ONNX
Qwen3-1.7B-Q4-ONNX is a machine learning model from Edison2ST. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This repository contains a highly optimized 4-bit quantized version of Qwen3-1.7B, specifically tailored for local inference on mid-range Android hardware.
Access
Public
Updated Jan 29, 2026
Repo size
2.8 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin1.4 GB · 99%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Jan 25, 2026
- Updated
- Jan 29, 2026
- SHA
- 3a528947
Base models
Model card
From the Hugging Face model README
Qwen3-1.7B-Q4 - Optimized 4-bit ONNX for Mobile
This repository contains a highly optimized 4-bit quantized version of Qwen3-1.7B, specifically tailored for local inference on mid-range Android hardware.
🚀 Mobile Optimization Highlights
Unlike standard exports, this version was built to maximize the throughput of the XNNPACK execution provider on ARM-based chipsets (like the Dimensity 6300).
- Memory Mapping (mmap): Configured for zero-copy loading using
MappedByteBuffer, allowing a 1.31GB model to run on devices with limited RAM without triggering the Android OOM (Out of Memory) killer. - Tensor Alignment: Optimized for block size 32 to ensure efficient cache line usage on Cortex-A76/A55 architectures.
📊 Performance Benchmark (MediaTek Dimensity 6300)
| Metric | Performance |
|---|---|
| Quantization | 4-bit Integer (Q4) |
| Inference Engine | ONNX Runtime (GenAI API) |
| Avg. Speed | ~10.89 tokens/sec (MediaTek Dimensity 6300). Surpassing Google's Edge Gallery on same hardware according to the Technical Report. |
| RAM Footprint | Stable ~1.31 GB |
🛠 Usage example
Run scripts/inference_example.py. Check the Ghost Assistant's Technical Report for more info and benchmarks.
🛠 Creating this ONNX File
Made it easier for you with a customized JSON file. Run the following command: olive run --config qwen_config.json with the model generated on the Technical Report.
- License
- apache-2.0
- Languages
- en, zh, multilingual
- Created
- Jan 25, 2026
- Updated
- Jan 29, 2026