Open-MOPD-SmolLM3-3B-RL-IF

Open-MOPD-SmolLM3-3B-RL-IF

This is the instruction-following (IF) domain teacher in the Open-MOPD pipeline. It starts from BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-MixSFT and is trained only on instruction-following prompts with verifiable rewards using GRPO. This release corresponds to training step 2,440.

Because instruction-following responses are shorter, training uses a 2,048-token response limit, rollout group size 8, and 3,000 planned steps. Other settings match the math and code teachers: global batch size 128, mini-batch size 32, learning rate 1e-6, no KL penalty, and accuracy-based group filtering.

Results

Results use n=1, temperature 1.0, enable_thinking=true, max_model_len=32768, top_p=0.95, top_k=-1, and stop_token_ids=[128012].

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-RL-IF"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")

Intended use and limitations

This is a domain teacher intended for distillation, not a general-purpose assistant. It was optimized only for instruction following and can perform worse than MixSFT on other domains.

Instruction following has the largest teacher gain in this pipeline (+9.62 points over MixSFT). It is also the domain most harmed by naive multi-teacher OPD: without token-budget balancing, it receives only about 1% of weighted gradient tokens.

Model specifications

• Architecture: SmolLM3ForCausalLM
• Parameters: approximately 3B
• Layers: 36
• Vocabulary size: 128,256
• Weights: BF16, approximately 6.2 GB
• Includes tokenizer and chat template