Downloads · 30 days
19
37% of all-time downloads
andresnowak/qwen38-27b-math-grpo-baseline-drgrpo-step100
qwen38-27b-math-grpo-baseline-drgrpo-step100 is a reinforcement learning model from andresnowak. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
Research checkpoint from an in-progress experiment on whether rewarding monitorable chain-of-thought changes how a model reasons. This is the control arm: trained on correctness alone. The matched treatment is qwen38-…
Downloads · 30 days
19
37% of all-time downloads
All-time downloads
51
Public
Parameters
27.4B
54.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors54.7 GB · 100%
From the Hugging Face model README
Research checkpoint from an in-progress experiment on whether rewarding monitorable chain-of-thought changes how a model reasons. This is the control arm: trained on correctness alone. The matched treatment is qwen38-27b-math-grpo-monitor-drgrpo-step100, identical except for a monitorability term in the reward.
This is not a tuned release. It is a 100-step RL checkpoint published so the two arms can be compared. See Limitations.
| Base model | Qwen/Qwen3.8-27B |
| Algorithm | GRPO with the Dr.GRPO correction (advantages not divided by group std) |
| Framework | NeMo RL |
| Data | AceReason-Math |
| Steps | 100 |
| Learning rate | 1e-6, cosine decay to 1e-7, 10 warmup iters |
| Weight decay | 0.0 |
| Parallelism | TP=4, PP=2 |
reward = correctness
No monitorability term. The treatment arm adds
× (1 + 0.1 × monitor_score), where the score comes from a weak monitor model
(HuggingFaceTB/SmolLM3-3B) reading the reasoning trace.
Batch shape is 16 prompts x 8 generations = 128 samples per step, over 100 steps.
| per sample | per step | total (100 steps) | |
|---|---|---|---|
| Total tokens (prompt + generated) | 4026 | ~515K | ~51.5M |
| Generated tokens only | 3873 | ~496K | ~49.6M |
Means are averaged over all 100 steps; prompts account for the ~153-token difference between the two rows.
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
m = "andresnowak/qwen38-27b-math-grpo-baseline-drgrpo-step100"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForCausalLM.from_pretrained(m, dtype=torch.bfloat16, device_map="auto")