Downloads · 30 days
26
13% of all-time downloads
Kiffaz11/Qwen3-0.6b-ifrlvr-grpo
Qwen3-0.6b-ifrlvr-grpo is a text generation model from Kiffaz11. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
An RLVR (GRPO) fine-tune of Qwen/Qwen3-0.6B for instruction following, trained against verifiable rewards only — IFEvalG programmatic constraint checkers plus GSM8K exact-answer match. Instruction following improves s…
Downloads · 30 days
26
13% of all-time downloads
All-time downloads
195
Public
Parameters
596M
14.3 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors2.4 GB · 100%
From the Hugging Face model README
An RLVR (GRPO) fine-tune of Qwen/Qwen3-0.6B for instruction following, trained against verifiable rewards only — IFEvalG programmatic constraint checkers plus GSM8K exact-answer match. Instruction following improves substantially and significantly in both reasoning modes, at no measurable cost to general knowledge.
Same-harness comparison against the original Qwen/Qwen3-0.6B. IFBench and IFEval
are prompt-level accuracy (strict / loose).
2048-token budget. GSM8K and MMLU use standard few-shot prompting with no chat template.
| Benchmark | base | grpo-C | Δ |
|---|---|---|---|
| IFBench (strict / loose) | 16.67 / 22.00 | 23.33 / 27.67 | +6.67 / +5.67 |
| IFEval (strict / loose) | 31.05 / 31.98 | 31.98 / 33.64 | +0.93 / +1.66 |
| GSM8K | 41.55 | 46.47 | +4.92 |
| MMLU | 39.97 | 40.00 | +0.03 |
Chat template (which enables Qwen3's reasoning block), 8192-token budget.
| Benchmark | base | grpo-C | Δ |
|---|---|---|---|
| IFBench (strict / loose) | 20.33 / 26.00 | 26.33 / 31.33 | +6.00 / +5.33 |
| IFEval (strict / loose) | 57.30 / 60.81 | 64.70 / 69.13 | +7.40 / +8.32 |
| GSM8K | 62.93 | 60.42 | −2.51 (n.s.) |
| MMLU (zero-shot CoT) | 47.51 | 45.82 | −1.69 (n.s.) |
Paired McNemar exact test on stored per-prompt outputs, with a paired bootstrap
confidence interval (20 000 resamples). Benchmarks scored through lm_eval have no
per-item log, so they use a conservative unpaired two-proportion test.
| Comparison | Δ | 95% CI | p |
|---|---|---|---|
| IFBench strict · non-thinking | +6.67 | [+3.33, +10.33] | 3.3e-04 |
| IFBench strict · thinking | +6.00 | [+2.00, +10.33] | 7.9e-03 |
| IFEval strict · thinking | +7.39 | [+3.33, +11.65] | 5.7e-04 |
| IFEval loose · thinking | +8.13 | [+3.88, +12.38] | 1.8e-04 |
| GSM8K · non-thinking | +4.92 | [+1.14, +8.70] | 0.011 |
| MMLU · non-thinking | +0.03 | [−1.12, +1.18] | 0.959 |
| GSM8K · thinking | −2.51 | [−6.22, +1.20] | 0.185 |
| MMLU-CoT · thinking | −1.69 | [−5.93, +2.55] | 0.434 |
Every instruction-following gain is significant. Nothing regresses significantly.
flexible-extract. The strict-match filter
requires a literal #### n ending that a reasoning response never emits; it scores
formatting, not arithmetic (base 15.92 / grpo-C 9.33 under that filter).mmlu_flan_cot_zeroshot, 57 subjects, 1065 items). The default MMLU task is scored
by log-likelihood over the four options — nothing is generated, so "thinking" is
undefined for it. Both models use the identical subsample and seed.Harness calibration against published Qwen3-0.6B figures: our base measurement gives IFBench-loose 26.00 (published 25.67) and IFEval-strict 57.30 (published 59.89).
| Init | a task-arithmetic merge of two earlier GRPO runs (WARP-style restart, not from base) |
| Algorithm | GRPO (TRL), 1200 steps, 8 generations per prompt |
| Data | allenai/IF_multi_constraints_upto5 (80%) + GSM8K (20%) |
| Reward | IFEvalG constraint verifiers; GSM8K exact answer |
| KL β | 0.02 — realised KL mean 0.040, max 0.164 |
| Reward curve | 0.551 → 0.631, flat across the final 300 steps |
| Seed | 17 |
| Hardware | 1× H100, ~100 minutes |
Prompts were reward-density prefiltered and 8-gram decontaminated against the evaluation sets: 7420 probed → 5389 kept (941 saturated, 1090 hopeless, 580 decontamination drops). Prompts the policy already always solves or never solves are dropped, since neither produces gradient under GRPO's group-relative advantage.
Interpolating this checkpoint back toward base at α=0.75 reduced IFBench by 1.0 and GSM8K by 2.0, so the raw RL checkpoint is the one released.
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Kiffaz11/qwen3-0.6b-ifrlvr-grpo-C"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
messages = [{"role": "user", "content": "Write exactly three sentences about the sea."}]
text = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
Set enable_thinking=True for the reasoning mode measured above.