NAS-PO-GSPO-Qwen3-VL-8B

NAS-PO-GSPO-Qwen3-VL-8B

Qwen3-VL-8B-Instruct trained for 202 RLVR steps with GSPO and NAS-PO: Native Attention-Strategy Policy Optimization for Vision-Language Models.

The GSPO backbone uses sequence-level importance ratios with token-level correction. NAS-PO adds trajectory-level NAS-AS advantage scaling and positive-anchored AAC routing consolidation.

Training

• Data: ViRL39K, 38,870 examples
• Rollouts per prompt: 8
• Rollout / actor global batch: 384 / 128
• Maximum prompt / response: 4096 / 256 tokens
• Sampling: temperature 1.0, top-p 0.99
• GSPO token loss with sequence averaging and clipping 3e-4 / 4e-4
• AdamW bf16: learning rate 1e-6, weight decay 0.01
• NAS-AS drop quantile: 0.25
• AAC target loss ratio: 0.3
• Hardware: 8 NVIDIA H100 80 GB GPUs

Evaluation

The paper reports 66.88 overall mean Accuracy@8 over eight configurations from seven benchmarks. Accuracy@8 is mean response accuracy, not pass@8.

Usage

from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

repo = "ANke121/NAS-PO-GSPO-Qwen3-VL-8B"
model = Qwen3VLForConditionalGeneration.from_pretrained(repo, torch_dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained(repo)

This is a research checkpoint. Outputs may be incorrect; verify them before consequential use. Citation metadata will be added after the paper is public.