Qwen3-8B-Base-Math-SeaSFT-Search-EOPD-Tau

Qwen3-8B-Base-Math-SeaSFT-Search-EOPD-Tau

An 8B agentic model trained with Entropy-Aware On-Policy Distillation (EOPD) in a multi-stage RL pipeline, built on Qwen3-8B-Base. This checkpoint is the final iteration (iter_0000300) of the Search-domain EOPD run.

Training pipeline

Method: Entropy-Aware On-Policy Distillation (EOPD)

EOPD combines two KL directions to get the best of mode-seeking and mode-covering distillation:

• Reverse-KL OPD everywhere (mode-seeking): the sampled-token log-ratio student_logp − teacher_logp is subtracted from the advantage, sharpening the student toward the teacher's modes.
• Forward-KL on high-entropy teacher tokens (mode-covering): where reverse KL alone collapses diversity, a differentiable forward-KL loss KL(p̃_teacher_topk ‖ p̃_student_topk) is added, gated by the teacher's per-token entropy (1[H_teacher > τ]) and computed over the teacher's renormalized top-k (k=16) distribution.

The teacher is served with SGLang and returns both the scalar sampled-token logprob (reverse KL) and the per-token top-k distribution + entropy (forward KL). Training was done with the slime RL framework on Megatron-LM; this repo is the Hugging Face safetensors export of the final checkpoint.

Architecture

Identical to Qwen3-8B-Base: 36 layers, hidden size 4096, 32 attention heads, 8 KV heads (GQA), head dim 128, QK-LayerNorm, RMSNorm, SwiGLU, RoPE (θ=1e6), 151,936 vocab, 32,768 context.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "willhx/Qwen3-8B-Base-Math-SeaSFT-Search-EOPD-Tau"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")

prompt = "Who won the Nobel Prize in Physics in 1921?"
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256)
print(tok.decode(out[0], skip_special_tokens=True))

The model is trained for a search/tool-use agent loop (Search-R1 style retrieval); to reproduce the agentic behavior, drive it with the same tool prompt/format used during training.

License

Apache-2.0, inherited from Qwen3-8B-Base.