Qwen3-VL-30B-A3B

Qwen3-VL-30B-A3B for SpacemiT K1/K3

This is a SpacemiT deployment package for the Qwen3-VL 30B-A3B mixture-of-experts vision-language model. The upstream model is from the Qwen Team and is designed for multimodal understanding across text, images, and video, including visual question answering, document understanding, OCR-style reading, coding, and agent workflows. See the Qwen3-VL Technical Report and the official Qwen3-VL announcement.

The package contains a Q4_1 GGUF text decoder and an ONNX vision encoder. The ONNX file is renamed to qwen3_vl_vision.onnx for a short, stable deployment name. The original Qwen3-VL model and this deployment package use the Apache-2.0 license; runtime dependency licenses remain with their upstream projects.

Files

• qwen3vl-30b-text-q4_1.gguf: Q4_1 GGUF text/MoE decoder.
• qwen3_vl_vision.onnx: ONNX vision encoder.
• configs/K1/config.json and configs/K3/config.json: platform-specific SpaceMIT EP settings.

Prerequisites

Install the SpacemiT ONNX Runtime release and build or unpack the SMT-enabled SpacemiT llama.cpp. Prebuilt RISC-V packages can be unpacked directly. To build llama.cpp from source, clone recursively, set RISCV_ROOT_PATH and SPACEMIT_ORT_DIR, then run:

bash build_spacemit.sh

On the board, set the runtime library path before starting the server:

export MODEL_DIR=/path/to/Qwen3-VL-30B-A3B-SpacemiT
export LLAMA_DIR=/path/to/llama.cpp-installed
export ORT_DIR=/path/to/spacemit-ort.riscv64.2.0.6
export LD_LIBRARY_PATH="$LLAMA_DIR/lib:$ORT_DIR/lib:${LD_LIBRARY_PATH:-}"

Supported platforms and core affinity

K1 uses AI cores 0;1;2;3 and four threads. Use configs/K1/config.json and -t 4.

K3 uses AI cores 8;9;10;11;12;13;14;15 and eight threads. Use configs/K3/config.json and -t 8.

Do not interchange the configuration files: each ep_config pins the SpaceMIT Execution Provider to the correct board's AI cores.

Run on board

K1:

"$LLAMA_DIR/bin/llama-server" \
  -m "$MODEL_DIR/qwen3vl-30b-text-q4_1.gguf" \
  --media-backend smt \
  --smt-config-dir "$MODEL_DIR/configs/K1" \
  -t 4 --host 0.0.0.0 --port 8080 --warmup

K3 uses the same command with configs/K3 and -t 8.

Send an OpenAI-compatible POST /v1/chat/completions request containing an image data URL and a prompt such as Describe the image content.. Set temperature to 0, use a small max_tokens value for a smoke test, and disable Qwen thinking with chat_template_kwargs: {"enable_thinking": false}.

Example request:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":[{"type":"text","text":"Describe the image content."},{"type":"image_url","image_url":{"url":"data:image/jpeg;base64,<BASE64_IMAGE>"}}]}],"max_tokens":64,"temperature":0,"chat_template_kwargs":{"enable_thinking":false}}'

The included humanspeech.jpg can be used for a basic end-to-end smoke test. Replace <BASE64_IMAGE> with its base64-encoded contents. The response should be HTTP 200 with a natural-language description; this is a functional check, not an accuracy benchmark.