Downloads · 30 days
30
38% of all-time downloads
pisces312-hf/qwen3-0.6b-mnn-qnn-npu
qwen3-0.6b-mnn-qnn-npu is a machine learning model from pisces312-hf. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Downloads · 30 days
30
38% of all-time downloads
All-time downloads
80
Public
Repo size
656 MB
Likes
0
Public
Click a slice to open those files.
.bin651 MB · 99%
From the Hugging Face model README
This is a QNN (Qualcomm Neural Network) offline model of Qwen3-0.6B, optimized for running on Qualcomm NPU (Hexagon HTP) via the MNN inference engine.
Key advantages over CPU inference:
| Item | Value |
|---|---|
| Base Model | Qwen3-0.6B |
| Quantization | W4A16 (4-bit weight, 16-bit activation) |
| Quantization Method | HQI + Sym + Omni |
| Block Size | 64 |
| Target SoC | SM8850 (Snapdragon 8 Elite Gen5, HTP V81) |
| MNN Version | 3.6.0 |
| QNN SDK | 2.39.0 |
| Metric | CPU fp16 (4T) | QNN HTP (NPU) |
|---|---|---|
| Prefill (20 tok) | ~306 t/s | 12.8~20.7 t/s |
| Decode | ~70 t/s | ~32 t/s |
Note: For small models (≤1B), CPU is faster in raw speed. NPU's advantage is power efficiency and freed CPU resources. For larger models (4B+), NPU becomes faster due to CPU bandwidth bottlenecks.
qwen3_0_6b_npu_noc4/
├── config.json # MNN LLM config (backend=cpu, llm_model=qnn/llm.mnn)
├── llm_config.json # Model architecture config
├── tokenizer.mtok # Tokenizer
├── embeddings_bf16.bin # CPU embedding layer (~297MB)
├── llm.mnn # CPU model graph (for reference)
├── llm.mnn.json # Model graph JSON
├── export_args.json # Export parameters
└── qnn/
├── llm.mnn # QNN plugin op descriptor (~2KB)
└── graph0.bin # HTP context binary with all weights (~325MB)
Note: llm.mnn.weight is NOT needed — all weights are compiled into qnn/graph0.bin. Set "use_external_weight": false in config.json (already set in this repo).
/sdcard/mnn-models/qwen3_0_6b_npu_noc4/# Build MNN with QNN support
cmake .. -DMNN_BUILD_LLM=ON -DMNN_QNN=ON -DQNN_SDK_ROOT=/path/to/qairt
# Run inference
./llm_demo /path/to/qwen3_0_6b_npu_noc4/config.json prompt.txt
llmexport.py with --generate_for_npu --seperate_embed --disable_transformer_c4 --quant_bit 4 --quant_block 64 --act_bit=16 --sym --omni --hqqgenerate_llm_qnn.py to compile into QNN HTP context binary (soc_id=87, dsp_arch=v81)This model is based on Qwen3-0.6B and follows the Apache 2.0 license.
这是 Qwen3-0.6B 的 QNN (Qualcomm Neural Network) 离线模型,通过 MNN 推理引擎在高通 NPU(Hexagon HTP)上运行。
相比 CPU 推理的优势:
| 项目 | 值 |
|---|---|
| 基础模型 | Qwen3-0.6B |
| 量化方式 | W4A16(4-bit 权重,16-bit 激活) |
| 量化方法 | HQI + Sym + Omni |
| 块大小 | 64 |
| 目标 SoC | SM8850(骁龙 8 Elite Gen5,HTP V81) |
| MNN 版本 | 3.6.0 |
| QNN SDK | 2.39.0 |
| 指标 | CPU fp16 (4T) | QNN HTP (NPU) |
|---|---|---|
| Prefill (20 tok) | ~306 t/s | 12.8~20.7 t/s |
| Decode | ~70 t/s | ~32 t/s |
注意:小模型(≤1B)CPU 纯速度更快。NPU 的优势在于低功耗和释放 CPU 资源。大模型(4B+)因 CPU 带宽瓶颈,NPU 会更快。
qwen3_0_6b_npu_noc4/
├── config.json # MNN LLM 配置(backend=cpu, llm_model=qnn/llm.mnn)
├── llm_config.json # 模型架构配置
├── tokenizer.mtok # 分词器
├── embeddings_bf16.bin # CPU embedding 层(~297MB)
├── llm.mnn # CPU 模型图(参考用)
├── llm.mnn.json # 模型图 JSON
├── export_args.json # 导出参数
└── qnn/
├── llm.mnn # QNN plugin op 描述(~2KB)
└── graph0.bin # HTP context binary,含全部权重(~325MB)
注意:不需要 llm.mnn.weight——所有权重已编译进 qnn/graph0.bin。config.json 中已设置 "use_external_weight": false。
/sdcard/mnn-models/qwen3_0_6b_npu_noc4/# 编译带 QNN 支持的 MNN
cmake .. -DMNN_BUILD_LLM=ON -DMNN_QNN=ON -DQNN_SDK_ROOT=/path/to/qairt
# 运行推理
./llm_demo /path/to/qwen3_0_6b_npu_noc4/config.json prompt.txt
llmexport.py,参数 --generate_for_npu --seperate_embed --disable_transformer_c4 --quant_bit 4 --quant_block 64 --act_bit=16 --sym --omni --hqqgenerate_llm_qnn.py 编译为 QNN HTP context binary(soc_id=87, dsp_arch=v81)本模型基于 Qwen3-0.6B,遵循 Apache 2.0 许可证。