Downloads · 30 days
5
4% of all-time downloads
assn2-dpo-best
hwanuk16/assn2-dpo-best
assn2-dpo-best is a text generation model from hwanuk16. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama3.2.
CAS4133 (Yonsei) Assignment 2 submission. DPO-trained Llama-3.2-1B-Instruct for GSM8K mathematical reasoning.
All-time downloads
141
Public
Parameters
1.5B
3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3 GB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama3.2
- Model type
- llama
- Access
- Public
- Created
- May 21, 2026
- Updated
- May 21, 2026
- SHA
- ed533ce0
Try a prompt
Base models
Model card
From the Hugging Face model README
Assn2 DPO (Llama-3.2-1B + weak-strong preference data)
CAS4133 (Yonsei) Assignment 2 submission. DPO-trained Llama-3.2-1B-Instruct for GSM8K mathematical reasoning.
Pipeline
- SFT on
openai/gsm8k(Socratic). LR=1e-6, 3 epochs, QLoRA r=16. - DPO on weak-strong preference data (
hwanuk16/assn2-pref-v3):- chosen = Qwen2.5-Math-7B-Instruct (correct rollouts)
- rejected = base Llama-3.2-1B-Instruct (wrong rollouts)
- β=0.3, LR=1e-6, 1 epoch, bf16
- Reference model = SFT-tuned policy (frozen).
Evaluation
GSM8K test (100 problems, shuffled seed=42):
| Model | Accuracy |
|---|---|
| Base Llama-3.2-1B-Instruct | 0.51 |
| + SFT (sft-v5) | 0.56 |
| + DPO (this model) | 0.60 |
Limitations
- Trained only on GSM8K-style problems; may not generalize to other math benchmarks.
- 1B parameter base — much smaller than typical math LLMs (7B+).
- Best LR window is narrow: LR > 1e-6 leads to overfit collapse.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama3.2
- Languages
- en
- Created
- May 21, 2026
- Updated
- May 21, 2026