Downloads · 30 days
0
qwen3-0.6b-distill-r1
DrEntropy/qwen3-0.6b-distill-r1
qwen3-0.6b-distill-r1 is a machine learning model from DrEntropy. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Qwen3-0.6B (base) fine-tuned on DeepSeek-R1 math reasoning traces, following Chapter 8 of Raschka's Build a Reasoning Model (From Scratch).
Access
Public
Updated Aug 28, 2026
Repo size
1.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pth1.5 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Aug 28, 2026
- Updated
- Aug 28, 2026
- SHA
- 1775ed42
Base models
Datasets
Model card
From the Hugging Face model README
qwen3-0.6b-distill-r1
Qwen3-0.6B (base) fine-tuned on DeepSeek-R1 math reasoning traces, following Chapter 8 of Raschka's Build a Reasoning Model (From Scratch).
- Checkpoint:
qwen3-0.6B-distill-step06670-epoch2.pth— barestate_dictfor the book'sQwen3Model(reasoning_from_scratch.qwen3_batched), not a Hugging Face model. Load with the book's code + reasoning tokenizer. - Data:
deepseek-r1-math-train.jsonfromrasbt/math_distill - Script:
ch08/04_train_with_distillation/distill_batched.py - Args:
--epochs 2 --lr 1e-5 --max_seq_len 2048 --use_think_tokens --grad_clip 1.0 --batch_size 2 --validation_size 25 - Hardware: RTX PRO 6000 96 GB (RunPod), Aug 2026
distill_batched_metrics.csvhas train/val loss per 50 steps.
Evaluate with ch03/02_math500-verifier-scripts/evaluate_math500.py --which_model reasoning --checkpoint_path <this file>.
I only did 10 examples , at which it got 2 out of the 10 right.
- License
- apache-2.0
- Created
- Aug 28, 2026
- Updated
- Aug 28, 2026