Downloads · 30 days
11
29% of all-time downloads
lwaekfjlk/rft-webshop-strict-qwen2.5-7b-epoch3
rft-webshop-strict-qwen2.5-7b-epoch3 is a machine learning model from lwaekfjlk. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Rejection finetuning on only fully-correct WebShop trajectories. Companion to rft-webshop-qwen2.5-7b-epoch3, which used a relaxed threshold on the same rollout pool — the two differ in exactly one variable.
Downloads · 30 days
11
29% of all-time downloads
All-time downloads
38
Public
Parameters
7.6B
15.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors15.2 GB · 100%
From the Hugging Face model README
Rejection finetuning on only fully-correct WebShop trajectories. Companion to
rft-webshop-qwen2.5-7b-epoch3, which used a relaxed threshold on the same rollout
pool — the two differ in exactly one variable.
| strict (this model) | relaxed (rft-webshop-...-epoch3) | |
|---|---|---|
| Filter | reward == 1.0 | reward >= 0.6 |
| Trajectories | 395 (379 train / 16 val) | 4507 |
| Median kept reward | 1.00 | 0.71 |
| Distinct tasks | 279 | 2521 |
Same source pool: 14611 Qwen2.5-7B-Instruct rollouts on AgentGym webshop train split
(n=3 per task, T=1.0, max 30 rounds). Trajectory fingerprints confirm 100% overlap with
the pool the relaxed model was filtered from.
Full-parameter SFT, FSDP bf16, 3 epochs, lr 5e-6 (linear, 3% warmup), effective batch 16, max_seq_len 16384 (mean 2587 tokens, max 8383, 0% truncated), 47 steps/epoch. Loss: 0.398 -> 0.286 -> 0.248.
Measured on the 500-item test split (T=1.0), the relaxed model reproduces the reward distribution of its training data — lots of "good enough" purchases, few exact matches:
| mean reward | success (reward == 1.0) | |
|---|---|---|
| base Qwen2.5-7B | 0.318 | 4.0% |
| relaxed (>= 0.6) | 0.534 | 6.2% |
| an earlier strict run on the same pool | 0.514 | 15.8% |
Relaxing the threshold bought +0.02 mean reward and cost ~9.6pp of exact-match success. This checkpoint has not yet been evaluated under the same harness — the numbers above are the reason it exists, not a claim about it.
Checkpoint is bf16. Weights verified free of NaN/Inf and all-zero tensors before upload.