Qwen3.5-4B-rk3588

Qwen3.5-4B for RK3588 NPU

This repository provides a hardware-accelerated port of Qwen3.5-4B optimized for Rockchip RK3588 NPU.

<br>User:<image>Describe the image.<br><br> Answer: This is a surreal, humorous, and slightly absurd depiction of an astronaut relaxing on the Moon — likely in a fictional or satirical context.

Key elements:

• Astronaut: A person dressed in a bulky white spacesuit with a gold-tinted visor helmet is reclining casually on what appears to be a makeshift chair or cushion made from a green cooler box (possibly repurposed as seating). Their posture suggests complete relaxation — legs stretched out, one hand holding a green glass bottle.

• The Bottle: The astronaut is holding a clear green glass bottle with a label that reads “BrewDog” — a well-known Scottish craft beer brand. This adds to the comedic contrast: enjoying a beer in space? It’s impossible under current physics and regulations, but it fits the whimsical tone.

• Setting: The background shows the lunar surface — gray, cratered terrain with no atmosphere or clouds. In the distance, Earth hangs prominently in the blackness of space, dotted with stars. A ladder leans against a structure (perhaps part of a base or rover) on the right side.

• Tone & Style: The image blends realism (detailed spacesuit, realistic Moon surface and Earth) with fantasy and humor. It plays on pop culture tropes — astronauts drinking beer in space is often used for comedic effect in movies, cartoons, or memes.

Interpretation: This isn’t a real photograph — it’s digitally manipulated or conceptually imagined art meant to evoke amusement. It juxtaposes the solemnity of space exploration with the trivial pleasure of enjoying a drink, creating irony and levity.

In short: An astronaut lounging on the Moon, sipping BrewDog beer, with Earth watching over them. 🌕🍻🌍

---

Model Files

Hardware Requirements

• Rockchip RK3588 / RK3588S
• RKNPU2 driver
• Tested on:
  • Rock 5C
  • Ubuntu 22.04 / 24.04 (Joshua Riek)

Runtime Requirements

• RKLLM runtime (version 1.3.0)
• RKNN runtime (rknpu2)
• OpenCV (for image preprocessing)

Model performance benchmark (FPS)

All models, with C++ examples, can be found on the Q-engineering GitHub.<br><br> All LLM models are quantized to w8a8, while the VLM vision encoders use fp16.<br>

¹ The total used memory; LLM plus the VLM. <br> ² When an llm/vlm model is loaded for the first time from your disk to RAM or NPU, it is called a cold start.<br> The duration depends on your OS, I/O transfer rate, and memory mapping.<br> ³ Subsequent loading (warm start) takes advantage of the already mapped data in RAM. Mostly, only a few pointers need to be restored.<br><br> <img width="1000" height="700" alt="Plot_Tokens" src="https://github.com/user-attachments/assets/7342debe-769c-46c9-8a83-755caf7d67dc" /><br> <img width="1000" height="700" alt="PlotMemory" src="https://github.com/user-attachments/assets/cf4362e6-f644-46d3-9b74-d129b23d3c44" />

Example Usage

• see: https://github.com/Qengineering/Qwen3.5-4B-NPU

Notes

• KV Cache up to 16384
• This is not a Transformers-compatible model
• This repository provides precompiled NPU binaries
• CPU fallback is not supported