Downloads · 30 days
9
20% of all-time downloads
victoria-kp/qwen25vl-7b-simtoreal-sft-mixed
qwen25vl-7b-simtoreal-sft-mixed is a machine learning model from victoria-kp. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as mit.
SFT adapter trained on a 50/50 mix of Goodbuy (sim) and Bestbuy (real) tasks.
Downloads · 30 days
9
20% of all-time downloads
All-time downloads
46
Public
Repo size
773 MB
Likes
0
Public
Click a slice to open those files.
.safetensors762 MB · 98%
From the Hugging Face model README
SFT adapter trained on a 50/50 mix of Goodbuy (sim) and Bestbuy (real) tasks.
Part of the Halluminate blog post on sim-to-real transfer for web agents. Tasks: goodbuy-bestbuy-mirrored-tasks. Training logs: W&B report.
| Eval on | Goodbuy (sim) | Bestbuy (real) |
|---|---|---|
| This adapter | 32.0% | 11.0% |
from peft import PeftModel
from transformers import Qwen2_5_VLForConditionalGeneration
base = Qwen2_5_VLForConditionalGeneration.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")
model = PeftModel.from_pretrained(base, "victoria-kp/qwen25vl-7b-simtoreal-sft-mixed")
QLoRA (4-bit NF4 base, LoRA rank 64, alpha 128, attention + MLP layers) on success-only Claude Fable teacher trajectories from the mixed 128+128 training split. One example per agent step: observation in (screenshot, task instruction, BID element list, history), teacher reasoning and JSON action out. Learning rate 5e-5 cosine with 5% warmup, effective batch size 2, max sequence length 4096. This is the validation-loss-optimal checkpoint (step 1500, epoch 0.81).