Downloads · 30 days
30
8% of all-time downloads
SOD-0.6B
youngzhong/SOD-0.6B
SOD-0.6B is a text generation model from youngzhong. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
All-time downloads
363
Public
Parameters
752M
1.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.5 GB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- qwen3
- Access
- Public
- Created
- May 12, 2026
- Updated
- May 22, 2026
- SHA
- 0bc5ad87
Try a prompt
Base models
Model card
From the Hugging Face model README
<div align="center">
<h1>SOD-0.6B</h1>
<p>
<a href="https://arxiv.org/abs/2605.07725">
<img src="https://img.shields.io/badge/Paper-arXiv-red?logo=arxiv&logoColor=red" alt="Paper on arXiv"/>
</a>
<a href="https://github.com/YoungZ365/SOD">
<img src="https://img.shields.io/badge/Code-GitHub-black?logo=github&logoColor=white" alt="Code on GitHub"/>
</a>
<a href="https://huggingface.co/collections/youngzhong/sod-6a03530369d76913c24a4ffb">
<img src="https://img.shields.io/badge/Collection-SOD-yellow?logo=huggingface" alt="HuggingFace Collection"/>
</a>
<a href="https://huggingface.co/papers/2605.07725">
<img src="https://img.shields.io/badge/Daily%20Paper-SOD-yellow?logo=huggingface&logoColor=yellow" alt="HuggingFace Daily Paper"/>
</a>
<a href="https://www.alphaxiv.org/abs/2605.07725">
<img src="https://img.shields.io/badge/alphaXiv-2605.07725-purple?logo=arxiv&logoColor=white" alt="alphaXiv"/>
</a>
</p>
</div>
About
SOD-0.6B is a 0.6B student model distilled from a 4B teacher using SOD (Step-wise On-policy Distillation), a method designed for training small language model agents with tool-integrated reasoning capabilities.
SOD addresses the cascading error propagation problem in on-policy distillation for agentic reasoning by introducing an adaptive step-level weighting mechanism that suppresses distillation loss on drifted steps and restores supervision when the student recovers alignment — all at negligible additional computational cost.
Model Information
| Attribute | Value |
|---|---|
| Base Model | Qwen3-0.6B |
| Teacher Model | SOD-GRPO_teacher-4B |
| Training Pipeline | Cold-Start SFT → SOD (Step-wise On-policy Distillation) |
| Parameters | 0.6B |
Related Models
| Model | Description |
|---|---|
| SOD-0.6B | SOD-distilled 0.6B student (this model) |
| SOD-1.7B | SOD-distilled 1.7B student |
| SOD-GRPO_teacher-4B | GRPO-trained 4B teacher model |
Performance
We report average@32 over 5 runs on challenging math, science, and code benchmarks.
0.6B Student Results
| Method | AIME 2024 | AIME 2025 | GPQA-Diamond | LiveCodeBench-v6 | Average |
|---|---|---|---|---|---|
| Vanilla | 7.71 | 12.81 | 13.24 | 14.89 | 12.16 |
| SFT | 5.67 | 5.42 | 15.20 | 9.61 | 8.97 |
| GRPO | 4.06 | 4.90 | 20.38 | 15.95 | 11.32 |
| OPD | 16.82 | 22.95 | 17.76 | 22.65 | 20.04 |
| OPSD_gt | 12.63 | 17.04 | 17.32 | 16.73 | 15.93 |
| OPSD_hint | 9.77 | 14.12 | 15.98 | 12.65 | 13.13 |
| SOD (This Model) | 20.84 | 26.13 | 22.19 | 27.72 | 24.22 |
Teacher Model (4B)
| Method | AIME 2024 | AIME 2025 | GPQA-Diamond | LiveCodeBench-v6 | Average |
|---|---|---|---|---|---|
| GRPO | 67.60 | 60.42 | 55.19 | 63.13 | 61.59 |
Key Highlights
- 🏆 Strong 0.6B agent: Achieves 26.13% on AIME 2025 (average@32)
- 📈 +20.86% over second-best baseline (OPD) on average
- 💡 Minimal extra compute: The divergence metric reuses log-probabilities already computed in the forward pass
Citation
@article{zhong2026sod,
title={SOD: Step-wise On-policy Distillation for Small Language Model Agents},
author={Zhong, Qiyong and Zheng, Mao and Song, Mingyang and Lin, Xin and Sun, Jie and Jiang, Houcheng and Wang, Xiang and Fang, Junfeng},
journal={arXiv preprint arXiv:2605.07725},
year={2026}
}
- Task
- Text Generation
- Library
- transformers
- Type
- qwen3
- License
- apache-2.0
- Languages
- en
- Created
- May 12, 2026
- Updated
- May 22, 2026