qwen3_vl

Qwen3-VL: Optimized for AMD ROCm

Qwen3-VL-4B-Instruct is a dense vision-language model supporting text and image input, served via vLLM's OpenAI-compatible HTTP API. This repository packages inference for vision-language text generation using vLLM, exported and validated for AMD ROCm so it runs efficiently on AMD GPUs.

This is based on the implementation of Qwen3-VL found here. This repository contains configurations and scripts optimized for AMD® ROCm™ platforms. You can use the qwen3_vl AMD scripts to reproduce results or export with custom configurations. More details on model performance can be found here.

---

Task Overview

Task: Vision-language text generation (text + image)

Dataset: MMMU (vision, 30 subjects), MMLU (text, 57 subjects, 5-shot)

Output metrics: MMMU accuracy, MMLU accuracy (5-shot), per-modality latency/throughput

vLLM note: GPU-only (vLLM server). No CPU or NPU inference path is available. --enforce-eager is always on — CUDA graph capture produces degenerate output on gfx1150/1151 for vision models.

---

AMD ROCm Optimization

This model export has been adapted and validated for AMD Instinct™ / Radeon™ GPUs running ROCm. Key points:

• Validated backend: vLLM (ROCm-enabled, built from source, OpenAI-compatible HTTP API).
• No code changes required versus the upstream Qwen3-VL implementation — only environment/runtime configuration differs.
• No AMD NPU (VitisAI) backend for vLLM and no practical CPU path for a 4B-parameter generative VLM.

---

Getting Started

For setup instructions, evaluation scripts, and custom configuration options, see the qwen3_vl on GitHub.

---

Model Details

Model Type: Vision-language causal language model (dense)

Base Model: Qwen/Qwen3-VL-4B-Instruct

Model Stats:

• Context ceiling: 32768 tokens
• Precision tested: FP16 (default), BF16
• Weights: ~8 GB (GPU-resident)

---

Accuracy Pipeline

Higher accuracy means the model's answers agree more closely with the ground-truth choice — 100% would be perfect, and random chance depends on the number of answer options per question (typically 25% for 4-way MCQ). MMMU evaluates vision + multiple-choice reasoning across 30 subjects; MMLU evaluates text-only knowledge across 57 subjects via lm-evaluation-harness in loglikelihood mode.

Metrics Explained

Accuracy Results

Published results:

<!-- accuracy-table-start --> <!-- accuracy-table-end -->

---

Dig Deeper

Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?

📂 View the full project on GitHub

The GitHub repository includes:

• Setup and prerequisites for ROCm environments
• Scripts for building vLLM from source and server management
• Per-modality (text/image) latency and throughput benchmarking
• MMMU and MMLU accuracy evaluation and reproduction instructions