Downloads · 30 days
5
10% of all-time downloads
victoria-kp/qwen25vl-7b-simtoreal-grpo-mixed
qwen25vl-7b-simtoreal-grpo-mixed is a reinforcement learning model from victoria-kp. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as mit.
Best GRPO checkpoint of the mixed condition (round 5). The strongest adapter in the study on both sites.
Downloads · 30 days
5
10% of all-time downloads
All-time downloads
48
Public
Repo size
762 MB
Likes
0
Public
Click a slice to open those files.
.safetensors762 MB · 100%
From the Hugging Face model README
Best GRPO checkpoint of the mixed condition (round 5). The strongest adapter in the study on both sites.
Part of the Halluminate blog post on sim-to-real transfer for web agents. Tasks: goodbuy-bestbuy-mirrored-tasks. Training logs: W&B report.
| Eval on | Goodbuy (sim) | Bestbuy (real) |
|---|---|---|
| This adapter | 39.1% | 23.4% |
from peft import PeftModel
from transformers import Qwen2_5_VLForConditionalGeneration
base = Qwen2_5_VLForConditionalGeneration.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")
model = PeftModel.from_pretrained(base, "victoria-kp/qwen25vl-7b-simtoreal-grpo-mixed")
GRPO from the corresponding SFT adapter: 8 rollouts per task, 8 tasks per round, 10 rounds, binary matched-criterion reward from a deterministic page-state verifier, KL 0.03 to the frozen SFT adapter, clip 0.2, lr 5e-6, temperature 1.0. Trained on the mixed 128+128 split. This is the checkpoint with the best held-out success (round 5).