DASH-Qwen3-4B-LoRA

DASH-Qwen3-4B-LoRA

LoRA adapter for Qwen/Qwen3-4B, trained with DASH (Divergence-Adaptive Supervision Horizons), a credit-assignment method for on-policy self-distillation.

DASH replaces the uniform aggregation coefficient of vanilla on-policy self-distillation with adaptive propagation gates: each local distillation signal's gap to the sequence mean drives a backward multi-step recursion, giving every token a path-dependent weight instead of a flat 1/T.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen3-4B"
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "<org>/DASH-Qwen3-4B-LoRA")
tok = AutoTokenizer.from_pretrained("<org>/DASH-Qwen3-4B-LoRA")

msgs = [{"role": "user", "content": "Find the number of ordered pairs..."}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True, enable_thinking=True)
out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=38912,
                     do_sample=True, temperature=1.0, top_p=1.0)
print(tok.decode(out[0], skip_special_tokens=True))

To merge the adapter into the base weights: model = model.merge_and_unload().

Training

Trained on OpenThoughts-Math-30K (29,434 problems with reference solutions). The teacher is the same network with the LoRA adapter disabled, conditioned on the reference solution; the student sees only the problem and rolls out on-policy.

Code

Training and evaluation code: github.com/DBtxy/DASH-OPSD

Built on siyan-zhao/OPSD.

Citation

<!-- TODO -->

@inproceedings{TODO_key,
  title     = {DASH: Divergence-Adaptive Supervision Horizons for On-Policy
               Self-Distillation of Reasoning Models},
  author    = {TODO},
  year      = {2027}
}