Downloads · 30 days
7
29% of all-time downloads
shibuina/drawvla-verify-qwen3vl-2b-lora
drawvla-verify-qwen3vl-2b-lora is a image-text-to-text model from shibuina. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
LoRA adapter for Qwen/Qwen3-VL-2B-Instruct trained as a sketch-prompt validator (DrawVLA-Verify): given a robot-scene image with a drawn circle (which object) and arrow (where), plus a name-free caption such as "put t…
Downloads · 30 days
7
29% of all-time downloads
All-time downloads
24
Public
Repo size
81.2 MB
Likes
0
Public
Click a slice to open those files.
.safetensors69.8 MB · 86%
From the Hugging Face model README
LoRA adapter for Qwen/Qwen3-VL-2B-Instruct trained as a sketch-prompt validator (DrawVLA-Verify): given a robot-scene image with a drawn circle (which object) and arrow (where), plus a name-free caption such as "put this there", decide whether the instruction is right (correct, consistent, sufficient) or wrong.
Trained on shibuina/drawvla-prompt-validation (LIBERO-grounded synthetic prompts; 14 corruption modes + grounding configurations). LoRA r=16, alpha=32, dropout 0.05, all attention/MLP projections, 3 epochs, lr 1e-4 cosine, 3% warmup, seed 17.
| split | accuracy | balanced acc | macro-F1 | MCC | AUROC |
|---|---|---|---|---|---|
| synthetic test | 0.905 | 0.869 | 0.880 | 0.750 | 0.960 |
| human-drawn sketches (280, manual labels) | 0.629 | – | – | 0.276 | 0.721 |
Recall on invalid prompts 0.939, on valid prompts 0.799. A geometry-only classifier on the same test split reaches 0.932 accuracy, so the synthetic benchmark is largely separable from sketch coordinates; treat the human-sketch numbers as the deployment estimate.
The adapter must be scored the way it was trained: the user turn is the image plus
"<VERDICT_INSTRUCTION>\n\nCaption: <caption>", and the verdict is chosen by the label-span log-likelihood of
verdict: right vs verdict: wrong (see training/vlm_training.py and training/train_local.py --evaluate-adapter
in the DrawVLA repository). Prompting it differently (e.g. a system prompt + bare next-token scoring) roughly halves the margins.
from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel
base = "Qwen/Qwen3-VL-2B-Instruct"
model = PeftModel.from_pretrained(AutoModelForImageTextToText.from_pretrained(base, dtype="bfloat16"), "shibuina/drawvla-verify-qwen3vl-2b-lora")
processor = AutoProcessor.from_pretrained(base)
Files: adapter_config.json, adapter_model.safetensors, chat_template.jinja, processor_config.json, run_config.json, test_generation_metrics.json, tokenizer.json, tokenizer_config.json, train_metrics.json, validation_generation_metrics.json.