Qwen3-VL-4B-Instruct

AXERA-TECH/Qwen3-VL-4B-Instruct

Qwen3-VL-4B-Instruct is a image-text-to-text model from AXERA-TECH. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.

This version of Qwen3-VL-2B-Instruct and Qwen3-VL-4B-Instruct have been converted to run on the Axera NPU using w8a16 quantization.

Downloads · 30 days

17% of all-time downloads

All-time downloads

206

Public

Repo size

6.6 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.axmodel5.8 GB · 88%

At a glance

Task: Image-Text-to-Text
Library: transformers
License: mit
Access: Public
Created: Oct 27, 2025
Updated: Sep 7, 2026
SHA: 052d3999

Try a prompt

Base models

Model card

From the Hugging Face model README

Qwen3-VL

This version of Qwen3-VL-2B-Instruct and Qwen3-VL-4B-Instruct have been converted to run on the Axera NPU using w8a16 quantization.

Compatible with Pulsar2 version: 5.0

Convert tools links:

For those who are interested in model conversion, you can try to export axmodel through the original repo :

Pulsar2 Link, How to Convert LLM from Huggingface to axmodel

AXera NPU HOST LLM Runtime

Support Platform

AX650
- AX650N DEMO Board
- M4N-Dock(爱芯派Pro)
- M.2 Accelerator card

Image Process

Chips	input size	image num	image encoder	ttft(168 tokens)	w8a16	CMM	Flash
AX650	384*384	1	236 ms	907 ms	4.3 tokens/sec	7.3GiB	7.9GiB

Video Process

Chips	input size	image num	image encoder	ttft(600 tokens)	w8a16	CMM	Flash
AX650	384*384	8	778 ms	2442 ms	4.3 tokens/sec	7.3GiB	7.9GiB

The DDR capacity refers to the CMM memory that needs to be consumed. Ensure that the CMM memory allocation on the development board is greater than this value.

How to use

Download all files from this repository to the device

Demo Run

Image understand demo

start tokenizer server for image understand demo

axllm from https://github.com/AXERA-TECH/ax-llm

axllm run Qwen3-VL-2B-Instruct-gptqmodel-4bit-ax650/

run image understand demo

input text

图片描述的内容是什么

input image

root@ax650:/mnt/sdcard/projects/llm# axllm run Qwen3-VL-2B-Instruct-gptqmodel-4bit-ax650/
09:45:26.953 INF Init:2700 | LLM init start
tokenizer_type = 3
huggingface tokenizer mode = gpt2_byte_bpe
 96% | ##############################   |  30 /  31 [21.99s<22.73s, 1.36 count/s] init post axmodel ok,remain_cmm(5969 MB)
09:45:48.978 INF Init:3023 | max_token_len : 1023
09:45:48.978 INF Init:3026 | kv_cache_size : 1024, kv_cache_num: 1023
09:45:48.978 INF init_groups_from_model:1827 | prefill_token_num : 128
09:45:48.978 INF init_groups_from_model:2075 | decode grp: 0, gid: 0, max_token_len : 1023
09:45:48.978 INF init_groups_from_model:2079 | prefill grp: 0, gid: 1, history_cap: 0, total_cap: 128, symbolic_cap: 1
09:45:48.978 INF init_groups_from_model:2079 | prefill grp: 1, gid: 2, history_cap: 128, total_cap: 256, symbolic_cap: 128
09:45:48.978 INF init_groups_from_model:2079 | prefill grp: 2, gid: 3, history_cap: 256, total_cap: 384, symbolic_cap: 256
09:45:48.978 INF init_groups_from_model:2079 | prefill grp: 3, gid: 4, history_cap: 384, total_cap: 512, symbolic_cap: 384
09:45:48.978 INF init_groups_from_model:2086 | prefill_max_token_num : 512
09:45:48.978 INF Init:27 | LLaMaEmbedSelector use mmap
100% | ################################ |  31 /  31 [21.99s<21.99s, 1.41 count/s] embed_selector init ok
09:45:55.206 INF Init:1647 | Qwen token ids: vision_start=151652 image_pad=151655 video_pad=151656 audio_pad=-1
09:45:55.207 INF Init:1704 | VisionModule init ok: type=Qwen3VL, tokens_per_block=144, embed_size=2048, out_dtype=fp32
09:45:55.207 INF Init:1710 | VisionModule deepstack enabled: layers=3
09:45:55.207 WRN Init:1713 | Vision preprocess backend: SimpleCV (OpenCV not found at build time; minor differences vs OpenCV are possible)
09:45:55.207 INF load_config:554 | load config:
09:45:55.207 INF load_config:554 | {
09:45:55.207 INF load_config:554 |     "enable_repetition_penalty": true,
09:45:55.207 INF load_config:554 |     "enable_temperature": true,
09:45:55.207 INF load_config:554 |     "enable_top_k_sampling": false,
09:45:55.207 INF load_config:554 |     "enable_top_p_sampling": true,
09:45:55.207 INF load_config:554 |     "penalty_window": 128,
09:45:55.207 INF load_config:554 |     "repetition_penalty": 1.1,
09:45:55.207 INF load_config:554 |     "temperature": 0.7,
09:45:55.207 INF load_config:554 |     "top_k": 20,
09:45:55.207 INF load_config:554 |     "top_p": 0.8
09:45:55.207 INF load_config:554 | }
09:45:55.207 INF Init:3138 | LLM init ok
Commands:
  /q, /exit  退出
  /reset     重置 kvcache
  /dd        删除一轮对话
  /pp        打印历史对话
Ctrl+C: 停止当前生成
VLM enabled: after each prompt, input media path (empty = text-only). Use "video:<frames_dir>" or "video:<video_file>[:<fps>]" for video (default fps=2), "audio:<file>" for audio.
----------------------------------------
prompt >> 图片描述的内容是什么
media >> /mnt/sdcard/projects/llm/Qwen3-VL-2B-Instruct-gptqmodel-4bit-ax650/01.jpg
09:46:59.858 INF EncodeForContent:2344 | Qwen-VL pixel_values[0] bytes=884736 min=6 max=219 (w=384 h=384 tp=2 ps=16 sm=2)
09:47:00.019 INF EncodeForContent:2403 | vision cache store: /mnt/sdcard/projects/llm/Qwen3-VL-2B-Instruct-gptqmodel-4bit-ax650/test.jpg
09:47:00.034 INF SetKVCache:3721 | decode_grpid:0 prefill_grpid:1 history_cap:0 total_cap:128 symbolic_cap:1 precompute_len:0 input_num_token:169 prefer_symbolic_group:0
09:47:00.034 INF SetKVCache:3743 | current prefill_max_token_num:512
09:47:00.060 INF SetKVCache:3759 | first run
09:47:00.093 INF Run:4287 | input token num : 169, prefill_split_num : 2
09:47:00.093 INF Run:4367 | prefill chunk p=0 history_len=0 grpid=1 kv_cache_num=0 input_tokens=128
09:47:00.244 INF Run:4367 | prefill chunk p=1 history_len=128 grpid=3 kv_cache_num=256 input_tokens=41
09:47:00.442 INF Run:4680 | ttft: 348.99 ms
这09:47:00.442 INF Run:4722 | VLM decode positions: rope_start=37 dense_kv_start=169
是一张室内空间的图片，可能是客厅或餐厅区域。房间的墙壁为暖黄色调，搭配深色木质地板和一些绿色的装饰元素。空间内有多种家具和电器，包括一个大尺寸的平板电视、一个带有透明玻璃柜的橱柜以及一个圆形的餐桌。房间内的自然光线透过窗户洒进来，营造出温馨而现代的氛围。

此外，墙上的挂画、墙上的灯具和室内的设计风格都显示出一种简洁而现代的装修风格。

09:47:09.200 NTC Run:5095 | hit eos,decode avg 11.88 token/s
09:47:09.200 INF GetKVCache:3698 | precompute_len:274, remaining:238 (tracked)

Video understand demo

start tokenizer server for image understand demo

axllm run Qwen3-VL-2B-Instruct-gptqmodel-4bit-ax650/

run video understand demo

input text

描述视频的内容

input video

./video

root@ax650:/mnt/sdcard/projects/llm# axllm run Qwen3-VL-2B-Instruct-gptqmodel-4bit-ax650/
09:44:04.332 INF Init:2700 | LLM init start
tokenizer_type = 3
huggingface tokenizer mode = gpt2_byte_bpe
 96% | ##############################   |  30 /  31 [4.15s<4.29s, 7.23 count/s] init post axmodel ok,remain_cmm(5969 MB)
09:44:08.515 INF Init:3023 | max_token_len : 1023
09:44:08.515 INF Init:3026 | kv_cache_size : 1024, kv_cache_num: 1023
09:44:08.515 INF init_groups_from_model:1827 | prefill_token_num : 128
09:44:08.515 INF init_groups_from_model:2075 | decode grp: 0, gid: 0, max_token_len : 1023
09:44:08.515 INF init_groups_from_model:2079 | prefill grp: 0, gid: 1, history_cap: 0, total_cap: 128, symbolic_cap: 1
09:44:08.515 INF init_groups_from_model:2079 | prefill grp: 1, gid: 2, history_cap: 128, total_cap: 256, symbolic_cap: 128
09:44:08.515 INF init_groups_from_model:2079 | prefill grp: 2, gid: 3, history_cap: 256, total_cap: 384, symbolic_cap: 256
09:44:08.515 INF init_groups_from_model:2079 | prefill grp: 3, gid: 4, history_cap: 384, total_cap: 512, symbolic_cap: 384
09:44:08.515 INF init_groups_from_model:2086 | prefill_max_token_num : 512
09:44:08.515 INF Init:27 | LLaMaEmbedSelector use mmap
100% | ################################ |  31 /  31 [4.15s<4.15s, 7.47 count/s] embed_selector init ok
09:44:08.913 INF Init:1647 | Qwen token ids: vision_start=151652 image_pad=151655 video_pad=151656 audio_pad=-1
09:44:08.913 INF Init:1704 | VisionModule init ok: type=Qwen3VL, tokens_per_block=144, embed_size=2048, out_dtype=fp32
09:44:08.913 INF Init:1710 | VisionModule deepstack enabled: layers=3
09:44:08.913 WRN Init:1713 | Vision preprocess backend: SimpleCV (OpenCV not found at build time; minor differences vs OpenCV are possible)
09:44:08.914 INF load_config:554 | load config:
09:44:08.914 INF load_config:554 | {
09:44:08.914 INF load_config:554 |     "enable_repetition_penalty": true,
09:44:08.914 INF load_config:554 |     "enable_temperature": true,
09:44:08.914 INF load_config:554 |     "enable_top_k_sampling": false,
09:44:08.914 INF load_config:554 |     "enable_top_p_sampling": true,
09:44:08.914 INF load_config:554 |     "penalty_window": 128,
09:44:08.914 INF load_config:554 |     "repetition_penalty": 1.1,
09:44:08.914 INF load_config:554 |     "temperature": 0.7,
09:44:08.914 INF load_config:554 |     "top_k": 20,
09:44:08.914 INF load_config:554 |     "top_p": 0.8
09:44:08.914 INF load_config:554 | }
09:44:08.914 INF Init:3138 | LLM init ok
Commands:
  /q, /exit  退出
  /reset     重置 kvcache
  /dd        删除一轮对话
  /pp        打印历史对话
Ctrl+C: 停止当前生成
VLM enabled: after each prompt, input media path (empty = text-only). Use "video:<frames_dir>" or "video:<video_file>[:<fps>]" for video (default fps=2), "audio:<file>" for audio.
----------------------------------------
prompt >> 描述视频的内容
media >> /mnt/sdcard/projects/llm/Qwen3-VL-2B-Instruct-gptqmodel-4bit-ax650/video/
09:44:38.482 INF EncodeForContent:2344 | Qwen-VL pixel_values[0] bytes=884736 min=0 max=255 (w=384 h=384 tp=2 ps=16 sm=2)
09:44:38.643 INF EncodeForContent:2403 | vision cache store: /mnt/sdcard/projects/llm/Qwen3-VL-2B-Instruct-gptqmodel-4bit-ax650/video/frame_0000.jpg
09:44:38.674 INF EncodeForContent:2344 | Qwen-VL pixel_values[0] bytes=884736 min=0 max=255 (w=384 h=384 tp=2 ps=16 sm=2)
09:44:38.834 INF EncodeForContent:2403 | vision cache store: /mnt/sdcard/projects/llm/Qwen3-VL-2B-Instruct-gptqmodel-4bit-ax650/video/frame_0008.jpg
09:44:38.864 INF EncodeForContent:2344 | Qwen-VL pixel_values[0] bytes=884736 min=0 max=255 (w=384 h=384 tp=2 ps=16 sm=2)
09:44:39.023 INF EncodeForContent:2403 | vision cache store: /mnt/sdcard/projects/llm/Qwen3-VL-2B-Instruct-gptqmodel-4bit-ax650/video/frame_0016.jpg
09:44:39.049 INF SetKVCache:3721 | decode_grpid:0 prefill_grpid:1 history_cap:0 total_cap:128 symbolic_cap:1 precompute_len:0 input_num_token:456 prefer_symbolic_group:0
09:44:39.049 INF SetKVCache:3743 | current prefill_max_token_num:512
09:44:39.075 INF SetKVCache:3759 | first run
09:44:39.076 INF Run:4287 | input token num : 456, prefill_split_num : 4
09:44:39.076 INF Run:4367 | prefill chunk p=0 history_len=0 grpid=1 kv_cache_num=0 input_tokens=128
09:44:39.230 INF Run:4367 | prefill chunk p=1 history_len=128 grpid=3 kv_cache_num=256 input_tokens=128
09:44:39.401 INF Run:4367 | prefill chunk p=2 history_len=256 grpid=4 kv_cache_num=384 input_tokens=128
09:44:39.584 INF Run:4367 | prefill chunk p=3 history_len=384 grpid=4 kv_cache_num=384 input_tokens=72
09:44:39.795 INF Run:4680 | ttft: 719.15 ms
这09:44:39.795 INF Run:4722 | VLM decode positions: rope_start=324 dense_kv_start=456
是一段关于两只小松鼠的视频。

画面中，两只小松鼠正站在一块布满岩石和泥土的山坡上，它们身体前倾，相互靠近。一只小松鼠面向镜头，另一只则侧身对着镜头，它们似乎在进行一场亲密的互动或是在进行某种游戏。

背景是连绵起伏的山峦，植被茂盛，天空晴朗。整个场景呈现出一种自然和谐的氛围。

虽然图像有些模糊，但可以清晰地看到两只小松鼠的毛色——它们的毛色以灰白色为主，并带有棕色的条纹。它们的耳朵、眼睛以及尾巴都清晰可见。

整体来看，这是一段关于两只小松鼠在山林间嬉戏玩耍的视频，画面充满了生机与活力。

09:44:53.252 NTC Run:5095 | hit eos,decode avg 12.11 token/s
09:44:53.252 INF GetKVCache:3698 | precompute_len:620, remaining:-108 (tracked)

Task: Image-Text-to-Text
Library: transformers
License: mit
Languages: en, zh
Created: Oct 27, 2025
Updated: Sep 7, 2026