Downloads · 30 days
6
8% of all-time downloads
qwen3vl-8b-robocasa-v3-active-observation
DorianAtSchool/qwen3vl-8b-robocasa-v3-active-observation
qwen3vl-8b-robocasa-v3-active-observation is a image-text-to-text model from DorianAtSchool. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for peft.
LoRA adapter fine-tuning Qwen/Qwen3-VL-8B-Instruct for the RoboCasa behavior-cloning task-VLM ("v3 active-observation") pipeline: the model predicts the acting agent and is trained with --train-get-image, i.e. it also…
All-time downloads
76
Public
Repo size
175 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors175 MB · 100%
At a glance
- Task
- Image-Text-to-Text
- Library
- peft
- Access
- Public
- Created
- Jul 22, 2026
- Updated
- Jul 22, 2026
- SHA
- fde30267
Try a prompt
Base models
Model card
From the Hugging Face model README
qwen3vl-8b-robocasa-v3-active-observation
LoRA adapter fine-tuning Qwen/Qwen3-VL-8B-Instruct
for the RoboCasa behavior-cloning task-VLM ("v3 active-observation") pipeline: the
model predicts the acting agent and is trained with --train-get-image, i.e. it
also learns to actively request observation images during rollout.
Training details
- Base model:
Qwen/Qwen3-VL-8B-Instruct - Method: LoRA SFT via PEFT, rank 16, alpha 32, dropout 0.05
- target modules:
q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- target modules:
- Attention: FlashAttention-2, image resolution 512, max sequence length 8192
- Hardware: 8x NVIDIA B200, single node,
accelerateMULTI_GPU (DDP) - Schedule: 3 epochs, per-device batch size 8, gradient accumulation 1
- Flags:
--predict-acting-agent --train-get-image
Results
| metric | value |
|---|---|
| train_loss | 0.0338 |
| eval_loss | 0.0263 |
| train_runtime | 2:06:15 (3 epochs) |
| train_samples_per_second | 18.39 |
Logged to Weights & Biases: robocasa-bc-task-vlm/l923azcv
Usage
from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel
base = AutoModelForImageTextToText.from_pretrained(
"Qwen/Qwen3-VL-8B-Instruct", torch_dtype="bfloat16", device_map="auto",
attn_implementation="flash_attention_2",
)
model = PeftModel.from_pretrained(base, "DorianAtSchool/qwen3vl-8b-robocasa-v3-active-observation")
processor = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-8B-Instruct")
Framework versions
- PEFT 0.19.1
- transformers 5.13.0
- Task
- Image-Text-to-Text
- Library
- peft
- Created
- Jul 22, 2026
- Updated
- Jul 22, 2026