Downloads · 30 days
25
24% of all-time downloads
rspcunningham/ml-intern-qwen25-1.5b-gsm8k-20260828-1420
ml-intern-qwen25-1.5b-gsm8k-20260828-1420 is a text generation model from rspcunningham. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
LoRA adapter that improves Qwen/Qwen2.5-1.5B-Instruct on GSM8K-style grade-school math word problems. Trained autonomously by ml-intern on a single Modal A100-40GB inside a hard budget of 30 GPU-minutes / $3, using on…
Downloads · 30 days
25
24% of all-time downloads
All-time downloads
106
Public
Repo size
148 MB
Likes
0
Public
Click a slice to open those files.
.safetensors148 MB · 100%
From the Hugging Face model README
LoRA adapter that improves Qwen/Qwen2.5-1.5B-Instruct on GSM8K-style grade-school math word problems.
Trained autonomously by ml-intern on a
single Modal A100-40GB inside a hard budget of 30 GPU-minutes / $3, using only public data.
Baseline and adapter are scored by one harness — same prompt, same greedy decode, same answer
extractor, same 400 problems sampled (seed 1234) from openai/gsm8k main/test.
| model | GSM8K accuracy | #### format rate |
|---|---|---|
Qwen/Qwen2.5-1.5B-Instruct (untuned) | 56.25% | 30.75% |
| this adapter | 66.00% | 100.00% |
Paired over the same problems: 70 newly solved, 31 newly broken, net +39.
bAugmented-CoT SFT: MetaMathQA's rephrased/bootstrapped GSM rationales are longer and more varied, so they should fix the answer format without collapsing the richer chain-of-thought the instruct model already has.
| training data | meta-math/MetaMathQA (GSM_* rows) |
| train examples | 9576 |
| optimizer steps | 500 (effective batch 16, seq len 640) |
| LoRA | r=32, alpha=64, dropout=0.05, all attn + MLP projections |
| trainable params | 36,929,536 (2.39% of 1,543,714,304) |
| lr | 0.0001, 20 warmup steps then cosine |
| precision | bf16, sdpa attention |
| loss | completion-only (prompt masked to -100) |
| init / final train loss | 0.2529 / 0.1298 |
| GPU time, whole run | 19.9 min on 1x A100-40GB (~$0.70) |
Both candidate recipes got an identical step budget and identical hyperparameters, so the only
variable was the training data. See RESULTS.md for the losing path.
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct", dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "rspcunningham/ml-intern-qwen25-1.5b-gsm8k-20260828-1420")
prompt = tok.apply_chat_template(
[{"role": "user", "content": QUESTION + PROMPT_SUFFIX}],
tokenize=False, add_generation_prompt=True,
)
The adapter is trained to answer with a chain of thought ending in #### <number>. Use the exact
prompt suffix in load_test.py, which reproduces the eval harness end to end:
python load_test.py
Janet's ducks lay 16 eggs per day.
She eats three eggs for breakfast every morning, so there are 16 - 3 = 13 eggs left after breakfast.
She also bakes muffins for her friends every day with four eggs, so there are 13 - 4 = 9 eggs left after baking muffins.
She sells the remaining eggs at the farmers' market for $2 each, so she makes 9 * $2 = $18 every day at the farmers' market.
#### 18
TASK.md, RESEARCH.md, PLAN.md, BUDGET.md, EXPERIMENTS.md, RESULTS.md, VERIFY.md,
train.log, results.json and the full source (gsm8k_ft/) are bundled in this repo.
See RESULTS.md.