Downloads · 30 days
89
36% of all-time downloads
Qwen3-4B-Instruct-2507-RKLLM-v1.2.3
GatekeeperZA/Qwen3-4B-Instruct-2507-RKLLM-v1.2.3
Qwen3-4B-Instruct-2507-RKLLM-v1.2.3 is a machine learning model from GatekeeperZA. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for rkllm. The card lists the license as apache-2.0.
RKLLM conversion of Qwen/Qwen3-4B-Instruct-2507 for Rockchip RK3588 NPU inference.
All-time downloads
245
Public
Repo size
5.3 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.rkllm5.3 GB · 100%
At a glance
- Library
- rkllm
- License
- apache-2.0
- Access
- Public
- Created
- Aug 18, 2026
- Updated
- Aug 18, 2026
- SHA
- 395bd0ba
Base models
Model card
From the Hugging Face model README
Qwen3-4B-Instruct-2507 — RKLLM v1.2.3 (w8a8, RK3588)
RKLLM conversion of Qwen/Qwen3-4B-Instruct-2507 for Rockchip RK3588 NPU inference.
Converted with RKLLM Toolkit v1.2.3. The 2507 suffix denotes the July 2025 refresh of Qwen3-4B with improved instruction following.
Note: This model does not produce
<think>reasoning blocks in this RKLLM build (thinking mode is disabled). See RKLLM thinking mode limitations.
Key Details
| Property | Value |
|---|---|
| Base Model | Qwen/Qwen3-4B-Instruct-2507 |
| Toolkit Version | RKLLM Toolkit v1.2.3 |
| Runtime Version | RKLLM Runtime ≥ v1.2.1 (v1.2.3 recommended) |
| Quantization | w8a8, group size 128 |
| Target Platform | RK3588 |
| NPU Cores | 3 |
| Max Context Length | 16384 tokens |
| Optimization Level | 1 |
| Hybrid Ratio | 0.0 |
| Thinking Mode | ❌ Disabled |
| Languages | English, Chinese (multilingual) |
Why This Model?
Qwen3-4B-Instruct-2507 is the July 2025 update to Alibaba's Qwen3-4B, with improved reasoning and instruction following. At 4B parameters it is the largest text-only model in this RK3588 lineup and handles complex prompts well despite the quantization.
The 16k context window (vs 8k on smaller models) enables long document summarisation and multi-turn conversations.
Hardware Tested
- Orange Pi 5 Plus — RK3588, 16GB RAM, Armbian Linux
- RKNPU driver 0.9.8
- RKLLM Runtime v1.2.3
Usage
With the RKLLM API Server
mkdir -p ~/models/Qwen3-4B-Instruct-2507
cd ~/models/Qwen3-4B-Instruct-2507
git lfs install && git clone https://huggingface.co/GatekeeperZA/Qwen3-4B-Instruct-2507-RKLLM-v1.2.3 .
Use with GatekeeperZA/RKLLM-API-Server.
With the official RKLLM API demo
git clone https://github.com/airockchip/rknn-llm.git
cd rknn-llm/examples/rkllm_api_demo
./build/rkllm_api_demo /path/to/Qwen3-4B-Instruct-2507-rk3588-w8a8_g128-opt-1-hybrid-ratio-0.0-16k.rkllm 8192 16384
File Listing
| File | Description |
|---|---|
Qwen3-4B-Instruct-2507-rk3588-w8a8_g128-opt-1-hybrid-ratio-0.0-16k.rkllm | Quantized model for RK3588 NPU |
Compatibility Notes
- Minimum runtime: RKLLM Runtime v1.2.1. v1.2.3 recommended.
- RKNPU driver: ≥ 0.9.6
- SoCs: RK3588 / RK3588S. Not compatible with RK3576 without reconversion.
- RAM: ~5.5GB loaded. Requires 8GB+ board (16GB recommended).
Acknowledgements
- Alibaba Qwen Team for Qwen3
- Rockchip / airockchip for the RKLLM toolkit and runtime
- Converted by GatekeeperZA
- Library
- rkllm
- License
- apache-2.0
- Languages
- en, zh
- Created
- Aug 18, 2026
- Updated
- Aug 18, 2026