Qwen-4b-base-RLSD

Qwen3-4B-Base RLSD

RLSD self-distillation reinforcement learning on the local math training split.

This repository contains the final merged Hugging Face checkpoint from global_step_100. The training checkpoint was saved from FSDP shards and merged to safetensors for this upload.

Training Method

• Policy loss mode: rlsd.
• Self-distillation uses reprompt feedback and token reweighting.
• Token reweighting: lambda 0.5, eps_w 0.2, decay steps 50.
• Advantage estimator remains grpo in the trainer config.
• Reward function: local math compute_score reward manager.
• Fine-tuning type: full-parameter FSDP training, not LoRA.

Training Hyperparameters

Training Score

The plot below shows critic/score/mean logged during training.

CSV data is included in training_score.csv.

Intended Use

This model is intended for internal research and analysis of math-focused RL fine-tuning methods. It has not been broadly safety evaluated for production use.

Limitations

The model was trained for 100 optimization steps on a local math dataset split. Reported scores are training-time reward/validation metrics from the same experiment setup and should not be treated as broad benchmark results.