Downloads · 30 days
18
100% of all-time downloads
qwen3-4b-thinking-glp
Samll/qwen3-4b-thinking-glp
qwen3-4b-thinking-glp is a machine learning model from Samll. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.
Paper: Efficient Reasoning Training Does Not Always Harm CoT Faithfulness and Monitorability
All-time downloads
18
Public
Repo size
144 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors132 MB · 89%
At a glance
- Library
- peft
- Access
- Public
- Created
- Sep 14, 2026
- Updated
- Oct 2, 2026
- SHA
- 1d8f48a9
Base models
Model card
From the Hugging Face model README
qwen3-4b-thinking-glp
Paper: Efficient Reasoning Training Does Not Always Harm CoT Faithfulness and Monitorability
Collection: Efficient Reasoning: CoT Faithfulness & Monitorability
Method: Within-group length penalty from Kimi k1.5 (Kimi Team, 2025)
Hyperparameters
| Base model | Qwen/Qwen3-4B-Thinking-2507 |
| Method | Within-Group Length Penalty (GLP) |
| Released as | LoRA adapter, step 150 |
| Length-reward weight / warm-up | 0.1 / 100 steps |
| LoRA | rank 16, alpha 32, dropout 0, all attention and MLP projections |
| Optimiser | AdamW, learning rate 1e-5, KL coefficient 0 |
| Batch | 32 prompts x 16 generations |
| Sampling | temperature 0.8, top-p 0.95 |
| Data | numina_amc_aime (PRIME Eurus-2-RL-Data), ~2,223 prompts |
| Checkpoint selection | AIME22, every 50 steps |
| Hardware | 2x NVIDIA GH200 |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Thinking-2507", torch_dtype="auto", device_map="auto")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Thinking-2507")
model = PeftModel.from_pretrained(base, "Samll/qwen3-4b-thinking-glp")
- Library
- peft
- Created
- Sep 14, 2026
- Updated
- Oct 2, 2026