Downloads · 30 days
14
20% of all-time downloads
mrzhao13/qwen3.5-2b-shopsimulator-grpo-rl500-1ep
qwen3.5-2b-shopsimulator-grpo-rl500-1ep is a image-text-to-text model from mrzhao13. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
This is the final Hugging Face checkpoint produced by one epoch of online GRPO on 500 ShopSimulator tasks. It starts from mrzhao13/qwen3.5-2b-shopsimulator-sft-512-1ep and uses Pi for multi-turn tool interaction throu…
Downloads · 30 days
14
20% of all-time downloads
All-time downloads
69
Public
Parameters
1.9B
3.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors3.8 GB · 99%
From the Hugging Face model README
This is the final Hugging Face checkpoint produced by one epoch of online GRPO
on 500 ShopSimulator tasks. It starts from
mrzhao13/qwen3.5-2b-shopsimulator-sft-512-1ep and uses Pi for multi-turn
tool interaction through a modified Slime and ShopSimulator integration.
The post-training workload is text-only. The checkpoint retains the original Qwen3.5 multimodal architecture, but visual capabilities were not trained or evaluated.
| Item | Value |
|---|---|
| Original base model | Qwen/Qwen3.5-2B |
| RL initialization | mrzhao13/qwen3.5-2b-shopsimulator-sft-512-1ep |
| RL task set | rl_500, 500 unique tasks |
| Epochs | 1 |
| Candidates per task | 4 |
| Candidates / optimizer steps | 2,000 / 100 |
| Rollout batch size / global batch size | 5 / 20 |
| Learning rate | 1e-6 |
| Advantage estimator | GRPO |
| KL loss coefficient / type | 0.001 / low_var_kl |
| Maximum model turns | 40 |
| Maximum context / response length | 16,384 / 2,048 |
| Maximum tokens per GPU | 12,288 |
| Non-zero reward-variance groups | 352 / 500 (70.4%) |
| Completed / truncated / aborted candidates | 1,881 / 119 / 0 |
| Training framework | Slime + Megatron-LM + SGLang + Pi |
| Hardware | One NVIDIA Pro 6000D 84 GB |
| Weight identity SHA-256 | 8eff796683fa32bf74af7378a64646eea190e71cb52e397de431e7da72463f18 |
All three checkpoints were evaluated with the same modified ShopSimulator
service, deterministic price generation, fixed official_test_200 task slice,
and one rollout per task. These are k=1 point estimates.
| Model | Positive-reward pass@1 | Strict-success pass@1 | mean@1 r_loose | mean@1 r_hard |
|---|---|---|---|---|
| Qwen3.5-2B | 2.0% | 0.0% | 0.004286 | 0.000000 |
| SFT-512 1 epoch | 72.5% | 10.5% | 0.389829 | 0.124417 |
| This GRPO checkpoint | 90.5% | 31.0% | 0.627786 | 0.354530 |
The RL checkpoint also achieved a 91.5% environment completion rate and a 40.0% right-product pass@1 rate on this fixed evaluation.
Use this repository as a drop-in replacement for Qwen/Qwen3.5-2B with a
recent Transformers version that supports Qwen3.5. Refer to the
official Qwen3.5-2B model card for
loading and inference examples, replacing the model ID with this repository.
ShopSimulator evaluation requires the companion Pi extension, prompt template,
patched environment and deterministic pricing behavior from the
pi-slime-shopsimulator source repository when it is published.
The original Qwen model, the SFT initialization and these derivative weights
are distributed under Apache-2.0. The original Qwen license is included in
LICENSE. This model card identifies the checkpoint as modified and does not
grant rights to third-party ShopSimulator task or environment content.