Downloads · 30 days
15
33% of all-time downloads
Qwen3.5-9B-ARORA-a0.1-gsm8k-mathalg
wckwan/Qwen3.5-9B-ARORA-a0.1-gsm8k-mathalg
Qwen3.5-9B-ARORA-a0.1-gsm8k-mathalg is a image-text-to-text model from wckwan. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers.
Qwen/Qwen3.5-9B fine-tuned with reinforcement learning from verifiable rewards using the ARORA-RLOO advantage estimator (alpha = 0.1), trained with verl v0.9.1. This is the final checkpoint (global step 100).
All-time downloads
45
Public
Parameters
9.4B
37.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors37.6 GB · 100%
At a glance
- Task
- Image-Text-to-Text
- Library
- transformers
- Model type
- qwen3_5
- Access
- Public
- Created
- Sep 21, 2026
- Updated
- Sep 21, 2026
- SHA
- 70bbd022
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3.5-9B-ARORA-a0.1-gsm8k-mathalg
Qwen/Qwen3.5-9B fine-tuned with reinforcement learning from verifiable rewards using the
ARORA-RLOO advantage estimator (alpha = 0.1), trained with verl v0.9.1.
This is the final checkpoint (global step 100).
Training
| Base model | Qwen/Qwen3.5-9B |
| Advantage estimator | arora_rloo, alpha = 0.1 |
| Data | 3,200 prompts, 50% GSM8K / 50% MATH (algebra) |
| Steps | 100 |
| Batch | 32 prompts x 8 rollouts, PPO mini-batch 16 prompts |
| Learning rate | 2e-6 |
| KL coefficient | 1e-3 |
| Loss aggregation | seq-mean-token-sum-norm |
| Max prompt / response length | 2,048 / 32,768 tokens |
| Hardware | 4 GPUs, FSDP2 |
Notes
- Weights are stored in float32 (the FSDP master weights, ~37.6 GB, single
model.safetensors).config.jsondeclaresbfloat16, sodtype="auto"and vLLM load it in bf16. - The vision tower is included unchanged from the base model; only text RL training was performed.
- License and usage terms follow the base model.
Usage
from transformers import AutoModelForImageTextToText, AutoProcessor
repo = "wckwan/Qwen3.5-9B-ARORA-a0.1-gsm8k-mathalg"
model = AutoModelForImageTextToText.from_pretrained(repo, dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained(repo)
- Task
- Image-Text-to-Text
- Library
- transformers
- Type
- qwen3_5
- Created
- Sep 21, 2026
- Updated
- Sep 21, 2026