Downloads · 30 days
14
5% of all-time downloads
YuyangZhao/skill1-alfworld-skill-only-qwen3.5-4b-step40
skill1-alfworld-skill-only-qwen3.5-4b-step40 is a text generation model from YuyangZhao. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
An RL checkpoint of Qwen3.5-4B trained with GRPO on ALFWorld under the Skill1 framework, in the skill-only ablation: the policy is updated only through the skill-selection steps (retrieval-query generation and skill r…
Downloads · 30 days
14
5% of all-time downloads
All-time downloads
305
Public
Parameters
5.2B
10.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors10.4 GB · 100%
From the Hugging Face model README
An RL checkpoint of Qwen3.5-4B trained with GRPO on ALFWorld under the Skill1 framework, in the skill-only ablation: the policy is updated only through the skill-selection steps (retrieval-query generation and skill re-ranking). The executor's environment actions are rolled out with the same model but are excluded from the gradient, so any improvement comes from skill selection / distillation and the growing skill library rather than from the executor learning the task.
| Base model | Qwen/Qwen3.5-4B (thinking mode disabled via enable_thinking=False) |
| Algorithm | GRPO, group size 8, 16 tasks per step, lr 1e-6, KL coef 0.01 |
| Trainable phases | query generation + re-rank (executor play steps excluded from the batch) |
| Rewards | task outcome (discounted back to the selection steps), re-rank NDCG x 0.3, distillation intrinsic reward x 0.3 |
| Skill library | top-k 3 retrieval, utility EMA alpha 0.05, 1865 skills at step 40 |
| Steps | 40 (checkpoint saved every 10 steps) |
| Hardware | 4 x NVIDIA L40S, vLLM 0.19.1 rollouts, FSDP actor |
| step | success rate with retrieval | success rate without retrieval |
|---|---|---|
| 0 (base) | 0.281 | 0.406 |
| 20 | 0.344 | 0.391 |
| 30 | 0.484 | 0.438 |
| 40 (this checkpoint) | 0.359 | 0.391 |
The executor's own success rate (without retrieval) stays flat, as intended for the ablation; the with-retrieval number depends on the skill library used at inference.
The checkpoint is a standard Qwen3_5ForConditionalGeneration model and loads with
transformers >= 5 or vLLM. It was trained with the chat template's thinking mode
turned off:
from transformers import AutoTokenizer, AutoModelForImageTextToText
repo = "YuyangZhao/skill1-alfworld-skill-only-qwen3.5-4b-step40"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForImageTextToText.from_pretrained(repo, dtype="bfloat16", device_map="auto")
messages = [{"role": "user", "content": "..."}]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
The ALFWorld prompts expect the model to reason inside <think> </think> and then
emit one admissible action inside <action> </action>; the skill-selection prompts
expect <query>...</query> and <rank>...</rank> outputs.
model.safetensors; the vision tower of the base
model is included unchanged (it is not used for ALFWorld).