Qwen3.5-2B-rk3588

Qwen3.5-2B for RK3588 NPU

This repository provides a hardware-accelerated port of Qwen3.5-2B optimized for Rockchip RK3588 NPU.

<br>User:<image>Describe the image.<br><br> Answer: This is a surreal, digitally manipulated image that blends science fiction with everyday life in an unexpected setting. Here’s a detailed description:

Main Subject:

• A person wearing a white astronaut suit reclines on what appears to be a lunar surface or spacecraft interior.
• Their legs are propped up against the edge of a green cooler box, which has faint graffiti-style writing (“SOMETHING”) visible on its side.
• In their right hand, they hold a bright green glass beer bottle — likely a craft beer — as if enjoying a drink in zero gravity.

Setting & Background:

• The environment is clearly extraterrestrial: gray, cratered terrain resembling the Moon’s surface.
• Behind the astronaut, Earth dominates the sky — its blue oceans and swirling white clouds are visible against the black void of space dotted with stars.
• To the right, part of a metallic ladder or structure leans into the frame, suggesting this might be inside a spacecraft or on a lunar module.

Atmosphere & Style:

• The lighting is dramatic — strong highlights on the astronaut’s helmet visor and suit, contrasting sharply with the dark background.
• There’s a sense of irony or absurdity: an astronaut relaxing with beer next to Earth, defying gravity and normal human behavior in space.
• The image has a hyper-realistic yet dreamlike quality, enhanced by digital effects like lens flares and atmospheric glow around Earth.

Overall Impression: It’s a humorous, thought-provoking piece that plays on the contrast between isolation in space and comfort found in mundane pleasures — drinking beer while floating above our own planet. It invites reflection on human resilience, curiosity, and the strange ways we adapt to extreme environments.

---

Model Files

Hardware Requirements

• Rockchip RK3588 / RK3588S
• RKNPU2 driver
• Tested on:
  • Rock 5C
  • Ubuntu 22.04 / 24.04 (Joshua Riek)

Runtime Requirements

• RKLLM runtime (version 1.3.0)
• RKNN runtime (rknpu2)
• OpenCV (for image preprocessing)

Model performance benchmark (FPS)

All models, with C++ examples, can be found on the Q-engineering GitHub.<br><br> All LLM models are quantized to w8a8, while the VLM vision encoders use fp16.<br>

¹ The total used memory; LLM plus the VLM. <br> ² When an llm/vlm model is loaded for the first time from your disk to RAM or NPU, it is called a cold start.<br> The duration depends on your OS, I/O transfer rate, and memory mapping.<br> ³ Subsequent loading (warm start) takes advantage of the already mapped data in RAM. Mostly, only a few pointers need to be restored.<br><br> <img width="1000" height="700" alt="Plot_Tokens" src="https://github.com/user-attachments/assets/7342debe-769c-46c9-8a83-755caf7d67dc" /><br> <img width="1000" height="700" alt="PlotMemory" src="https://github.com/user-attachments/assets/cf4362e6-f644-46d3-9b74-d129b23d3c44" />

Example Usage

• see: https://github.com/Qengineering/Qwen3.5-2B-NPU

Notes

• KV Cache up to 16384
• This is not a Transformers-compatible model
• This repository provides precompiled NPU binaries
• CPU fallback is not supported