Qwen2.5-Coder-14B-Instruct-Thinking

Fine-Tuned Model: Qwen/Qwen2.5-Coder-14B-Instruct on GAIR/LIMO

Overview

This model is a fine-tuned version of the base model Qwen/Qwen2.5-Coder-14B-Instruct. It was trained on a subset of problems from the GAIR/LIMO dataset, specifically focusing on 611 problems over 14 training epochs.

After testing more I found that the model does not always include reasoning, I will update with more epochs.

Warning! The model often goes into an endless chain of reasoning. Need to use phrase: "Structure your thoughts. Be attentive to details."

Training Details

• Base Model: Qwen/Qwen2.5-Coder-14B-Instruct
• Dataset: GAIR/LIMO (subset of 611 problems)
• Epochs: 14
• Training Limitations: The training was constrained by the computational resources available on my machine, which means I haven't yet conducted a thorough evaluation of the model's performance improvements.

Key Observations

During testing, the fine-tuned model demonstrated significant improvements in reasoning ability compared to the base model. It began to provide more coherent and accurate responses, avoiding the mistakes observed in the base model during my initial tests.

Next Steps

While preliminary results are promising, further evaluation is needed to assess the overall improvement in model quality. I encourage the community to test the model and share their findings. Your feedback will be invaluable in understanding the extent of the improvements.

Acknowledgments

Special thanks to the creators of the Qwen/Qwen2.5-Coder-14B-Instruct and GAIR/LIMO datasets for providing the foundational resources.