BRIDGE

BRIDGE: Bridging Reasoning In Distillation Gap Elimination via Structure-Aware Masking

Checkpoints for the SDM 2026 paper. BRIDGE is a three-stage curriculum that distills long Chain-of-Thought reasoning from a large teacher into a compact student while improving accuracy and compressing output length.

• 📄 Paper (arXiv): https://arxiv.org/abs/2602.17686
• 💻 Code: https://github.com/Applied-Machine-Learning-Lab/SDM2026_BRIDGE

⭐ Final model (reported in the paper)

qwen2.5-3b/stage3_rewrite_v2/final_model — Qwen2.5-3B trained with the full BRIDGE pipeline. On GSM8K it reaches 76.19% accuracy / 167 avg tokens (vs. Base 64.90% / 230), evaluated on the 1,319-sample test set.

from huggingface_hub import snapshot_download
snapshot_download("bowen0815/BRIDGE",
                  allow_patterns="qwen2.5-3b/stage3_rewrite_v2/final_model/*")

Repository layout

qwen2.5-3b/     Qwen2.5-3B: stage1, stage2 (best+final), stage3, stage3_rewrite_v2 (FINAL),
                stage3_sft, stage4, pure_sft, stage2_G4/G8 ablations
llama-3.2-3b/   Llama-3.2-3B: stage1/2/3 + baselines (pure_sft, short_cot_kd, mix_length_kd, superrl_sft)
qwen2.5-1.5b/   Qwen2.5-1.5B: stage1/2/3_rewrite + superrl (weaker; kept for completeness)
misc/           phase1_unified
archive/        archived completed stage-1 SFT + validation artifacts
data/           teacher (Ollama) GSM8K rollout: gsm8k_full_7473.jsonl (raw, 7460),
                original_cot_7k_clean.jsonl (filtered correct, 6170), filter_stats.json

Each model directory is a standard 🤗 Transformers checkpoint (full fine-tuning, not LoRA).

Results (GSM8K, Qwen2.5-3B)

Zero-shot transfer: SVAMP 83.33% / MATH-500 38.20%.

Citation

@article{yu2026curriculum,
  title={Curriculum Learning for Efficient Chain-of-Thought Distillation via Structure-Aware Masking and GRPO},
  author={Yu, Bowen and Wang, Maolin and Zhang, Sheng and Wang, Binhao and Wen, Yi and Gao, Jingtong and Liu, Bowen and Zhao, Zimo and Wang, Wanyu and Zhao, Xiangyu},
  journal={arXiv preprint arXiv:2602.17686},
  year={2026}
}

License: MIT.