Downloads · 30 days
10
19% of all-time downloads
qwen2.5-1.5b-sft-gpt54mini-math_cot
vukien2301/qwen2.5-1.5b-sft-gpt54mini-math_cot
qwen2.5-1.5b-sft-gpt54mini-math_cot is a machine learning model from vukien2301. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
SFT fine-tune of Qwen/Qwen2.5-1.5B-Instruct on vukien2301/ultrainteractmathcotgpt54mini (78,349 GPT-5.4-mini regenerated math CoT problems from UltraInteract).
All-time downloads
52
Public
Parameters
1.5B
3.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.1 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- qwen2
- Access
- Public
- Created
- May 17, 2026
- Updated
- May 17, 2026
- SHA
- 89988995
Base models
Model card
From the Hugging Face model README
vukien2301/qwen2.5-1.5b-sft-gpt54mini-math_cot
SFT fine-tune of Qwen/Qwen2.5-1.5B-Instruct on
vukien2301/ultrainteract_math_cot_gpt54mini
(78,349 GPT-5.4-mini regenerated math CoT problems from UltraInteract).
Training
- Format: raw completion (no chat template) —
"{question}\n{model_prediction}"+ EOS - DoRA r=128, alpha=256, dropout=0.05
- Target modules: q,k,v,o,gate,up,down
- LR 2e-5, 1 epoch, batch 8 × grad-accum 4 (eff. 32)
- Adapter merged into base via
peft.merge_and_unload() - Hardware: 1× B200, bf16, flash_attention_2
Evaluation (lm-evaluation-harness, vLLM backend)
| Task | Shots | Base Qwen2.5-1.5B-Instruct | This model |
|---|---|---|---|
| GSM8K (flex) | 5 | 59.89 | 65.13 |
| GSM8K (strict) | 5 | 54.74 | 38.74 |
| GSM-Plus (flex) | 5 | 39.56 | 45.61 |
| GSM-Plus (str) | 5 | 32.45 | 24.18 |
| MATH500 | 4 | 12.20 | 10.40 |
Strict-match drop is format drift (\boxed{} vs #### N), not capability loss — flex confirms.
Source
Built for EMNLP H2H-SD compound-pipeline experiments. See [github / paper link TBD].
- Type
- qwen2
- License
- apache-2.0
- Languages
- en
- Created
- May 17, 2026
- Updated
- May 17, 2026