MATH-LLaMA-2-7B-KG

LoRID: A Reasoning Distillation Method via Multi-LoRA Interaction

📃 Paper • 💻 Code • 🤗 HF Repo

Abstract

The models for "Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interaction" [IJCAI 2025].

Key Contributions

• We focus on the mathematical reasoning distillation task and propose a novel method LoRID, which draws inspiration from the human beings teaching and learning pattern.
• We introduce knowledge during data augmentation and propose multi-LoRA interaction during model distillation, which improves the student’s reasoning abilities.
• Experimental results show that with the interaction between System 1 and System 2, LoRID outperforms previous state-of-the-art approaches and can be easily and effectively integrated into any Chain-of-Thought distillation method.

Citation

If this work is helpful, please kindly cite as:

@misc{li2025largemodelsteachstudent,
      title={Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interaction}, 
      author={Xinhe Li and Jiajun Liu and Peng Wang},
      year={2025},
      eprint={2508.13037},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2508.13037}, 
}