Downloads · 30 days
7
16% of all-time downloads
victoria-kp/qwen25vl-7b-simtoreal-grpo-goodbuy
qwen25vl-7b-simtoreal-grpo-goodbuy is a reinforcement learning model from victoria-kp. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as mit.
Best GRPO checkpoint of the sim-only condition (round 9).
Downloads · 30 days
7
16% of all-time downloads
All-time downloads
44
Public
Repo size
762 MB
Likes
0
Public
Click a slice to open those files.
.safetensors762 MB · 100%
From the Hugging Face model README
Best GRPO checkpoint of the sim-only condition (round 9).
Part of the Halluminate blog post on sim-to-real transfer for web agents. Tasks: goodbuy-bestbuy-mirrored-tasks. Training logs: W&B report.
| Eval on | Goodbuy (sim) | Bestbuy (real) |
|---|---|---|
| This adapter | 25.0% | 0.6% |
from peft import PeftModel
from transformers import Qwen2_5_VLForConditionalGeneration
base = Qwen2_5_VLForConditionalGeneration.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")
model = PeftModel.from_pretrained(base, "victoria-kp/qwen25vl-7b-simtoreal-grpo-goodbuy")
GRPO from the corresponding SFT adapter: 8 rollouts per task, 8 tasks per round, 10 rounds, binary matched-criterion reward from a deterministic page-state verifier, KL 0.03 to the frozen SFT adapter, clip 0.2, lr 5e-6, temperature 1.0. Trained on the Goodbuy split. This is the checkpoint with the best held-out success (round 9).