Downloads · 30 days
29
38% of all-time downloads
andresnowak/qwen38-27b-math-grpo-monitor-drgrpo-step100
qwen38-27b-math-grpo-monitor-drgrpo-step100 is a reinforcement learning model from andresnowak. Use it for the reinforcement learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
Research checkpoint from an in-progress experiment on whether a model can be pushed to express its reasoning in a way a weak monitor finds readable, without giving up task ability. This is the treatment arm; the match…
Downloads · 30 days
29
38% of all-time downloads
All-time downloads
76
Public
Parameters
27.4B
54.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors54.7 GB · 100%
From the Hugging Face model README
Research checkpoint from an in-progress experiment on whether a model can be pushed to express its reasoning in a way a weak monitor finds readable, without giving up task ability. This is the treatment arm; the matched control is qwen38-27b-math-grpo-baseline-drgrpo-step100, trained identically except for the monitorability term.
This is not a tuned release. It is a 100-step RL checkpoint published so the two arms can be compared. See Limitations.
| Base model | Qwen/Qwen3.8-27B |
| Algorithm | GRPO with the Dr.GRPO correction (advantages not divided by group std) |
| Framework | NeMo RL |
| Data | AceReason-Math |
| Steps | 100 |
| Learning rate | 1e-6, cosine decay to 1e-7, 10 warmup iters |
| Weight decay | 0.0 |
| Parallelism | TP=4, PP=2 |
reward = correctness x (1 + 0.1 * monitor_score)
monitor_score comes from a weak monitor model (HuggingFaceTB/SmolLM3-3B)
reading the reasoning trace, weighted at 0.1.
The correctness factor is a floor, not the objective: it stops the model
trading away math ability in pursuit of the style term. What the monitor term is
meant to change is how the model talks - steering the reasoning toward what the
weak monitor finds likely - while accuracy holds. The control arm uses
reward = correctness alone and puts no pressure on style.
Batch shape is 16 prompts x 8 generations = 128 samples per step, over 100 steps.
| per sample | per step | total (100 steps) | |
|---|---|---|---|
| Total tokens (prompt + generated) | 4164 | ~533K | ~53.3M |
| Generated tokens only | 4011 | ~513K | ~51.3M |
Means are averaged over all 100 steps; prompts account for the ~153-token difference between the two rows.
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
m = "andresnowak/qwen38-27b-math-grpo-monitor-drgrpo-step100"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForCausalLM.from_pretrained(m, dtype=torch.bfloat16, device_map="auto")