Downloads · 30 days
29
5% of all-time downloads
Efficient_CoT_DeepSeek-R1-Distill-Qwen-7B
Jianshu001/Efficient_CoT_DeepSeek-R1-Distill-Qwen-7B
Efficient_CoT_DeepSeek-R1-Distill-Qwen-7B is a machine learning model from Jianshu001. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
We Jianshu She, Zhuohao Li, Zhemin Huang and Muqi Li fine-tuned DeepSeek-R1-Distill-Qwen-7B using GRPO (Gradient-Regularized Policy Optimization) to achieve over 75% compression in Chain of Thought (CoT) length on the…
All-time downloads
638
Public
Parameters
7.6B
15.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors15.2 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- qwen2
- Access
- Public
- Created
- Feb 24, 2025
- Updated
- Mar 11, 2025
- SHA
- ad3483ba
Base models
Model card
From the Hugging Face model README
Efficient CoT for DeepSeek-R1-Distill-Qwen-7B
We Jianshu She, Zhuohao Li, Zhemin Huang and Muqi Li fine-tuned DeepSeek-R1-Distill-Qwen-7B using GRPO (Gradient-Regularized Policy Optimization) to achieve over 75% compression in Chain of Thought (CoT) length on the MATH dataset, with less than 5% accuracy loss.
Results Comparison
| Model | Final Accuracy | Average CoT Length | Average Answer Length |
|---|---|---|---|
| Baseline (Full CoT) | 92.08% | 450.95 words | 481.19 words |
| Efficient_CoT_DeepSeek-R1-Distill-Qwen-7B | 89.11% | 113.06 words | 125.94 words |
Our optimization strategy significantly reduces CoT length while maintaining high accuracy, making inference more efficient. This approach is particularly suitable for resource-constrained environments without sacrificing reasoning performance.
- Type
- qwen2
- License
- apache-2.0
- Created
- Feb 24, 2025
- Updated
- Mar 11, 2025