Open-MOPD-SmolLM3-3B-Final

Open-MOPD-SmolLM3-3B-Final

This is the final flagship model from the Open-MOPD pipeline. Starting from BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-MixSFT, it is trained with multi-teacher online policy distillation using three domain-specific RL teachers for math, code, and instruction following. This release corresponds to training step 200.

Each prompt is hard-routed to its domain teacher. The dense reward is the teacher-student log-probability gap over the student's top-k distribution (k=16, nucleus p=0.99), weighted by the student probabilities and applied directly as the token-level advantage.

Open-MOPD uses three mechanisms to prevent cross-domain training imbalance:

• Token-share balancing keeps the weighted gradient-token share near one third per domain. Without it, math and code consume about 99% of gradient tokens.
• Gap-aware allocation (alpha=1) shifts the training budget toward domains with more remaining teacher-student improvement.
• Reward refresh recomputes the student-dependent part of the dense reward at every inner update without adding extra prefills.

Training uses a global batch size of 1,024, mini-batch size 256, constant learning rate 1.5e-6, clipping at 0.2/0.28, no KL penalty, and a domain sampling ratio of math:code:IF = 2:2:1.

Results

Per-dataset scores are AIME24 21.98, AIME25 22.86, LiveCodeBench v5 20.84, LiveCodeBench v6 22.63, IFEval 74.49, and IFBench_test 24.67. Recovery measures the fraction of the overall score gap between MixSFT and the routed-teacher upper bound recovered by the final student.

Evaluation protocol

Results are averaged per dataset, then per domain, followed by a macro-average over the three domains.

• Math: AIME24 and AIME25, avg@64, temperature 0.6.
• Code: LiveCodeBench v5 and v6, avg@10, temperature 1.0.
• Instruction following: IFEval and IFBench_test, n=1, temperature 1.0, with enable_thinking=true.

All evaluations use max_model_len=32768, top_p=0.95, top_k=-1, and stop_token_ids=[128012]. Code is reported as avg@10, not best@10.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "BytedTsinghua-SIA/Open-MOPD-SmolLM3-3B-Final"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype="bfloat16",
    device_map="auto",
)

messages = [{"role": "user", "content": "Find all real solutions of x^3 - 3x + 1 = 0."}]
inputs = tokenizer.apply_chat_template(
    messages,
    return_tensors="pt",
    add_generation_prompt=True,
    enable_thinking=True,
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=8192,
    temperature=0.6,
    top_p=0.95,
    do_sample=True,
)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

Single greedy generations are not comparable to the reported benchmark results; use the sampling protocol above for reproduction.

Model family

SmolLM3-3B-Base -> MixSFT initialization -> Math/Code/IF RL teachers -> Open-MOPD Final.

The full family and training/evaluation data are released under the BytedTsinghua-SIA organization.

Model specifications

• Architecture: SmolLM3ForCausalLM
• Parameters: approximately 3B
• Layers: 36
• Vocabulary size: 128,256
• Weights: BF16, approximately 6.2 GB
• Context used in evaluation: 32,768 tokens
• Includes tokenizer and chat template