math-tutor-qwen-lora

Math Tutor - Qwen (LoRA)

Model Description

Math Tutor - Qwen is a QLoRA fine-tuned version of Qwen2.5-1.5B-Instruct trained on the GSM8K dataset. The objective is to improve multi-step mathematical reasoning for grade-school arithmetic word problems.

This repository contains only the LoRA adapter weights and must be loaded together with the original Qwen2.5-1.5B-Instruct base model.

---

Model Details

• Developed by: Moiz Baloch
• Model type: LoRA Adapter (PEFT)
• Base model: Qwen/Qwen2.5-1.5B-Instruct
• Language: English
• License: MIT
• Fine-tuning method: QLoRA (4-bit)

---

Intended Use

This model is intended for:

• Mathematical reasoning
• Step-by-step arithmetic solutions
• Educational tutoring
• Grade-school math assistance
• Learning demonstrations for LLM fine-tuning

---

Out-of-Scope Use

This model is not intended for:

• Medical advice
• Legal advice
• Financial advice
• High-stakes decision making
• General knowledge evaluation

---

Training Dataset

Dataset:

GSM8K (Grade School Math 8K)

• 7,473 training samples
• 1,319 evaluation samples

Dataset: https://huggingface.co/datasets/openai/gsm8k

---

Training Procedure

Fine-tuning Method

• QLoRA
• 4-bit Quantization
• PEFT

Hyperparameters

• Epochs: 2
• Learning Rate: 2e-4
• Batch Size: 2
• Gradient Accumulation: 4
• Optimizer: paged_adamw_8bit

---

Hardware

Training performed on:

• Kaggle Notebook
• NVIDIA Tesla T4 GPU (16 GB)

---

Evaluation

The model was evaluated on the GSM8K evaluation split during training.

Future work includes comparison against the base Qwen model on unseen mathematical reasoning tasks.

---

Limitations

The model has only been fine-tuned on GSM8K.

It may perform poorly on:

• Advanced mathematics
• Symbolic algebra
• Geometry proofs
• Scientific reasoning
• Non-English problems

---

Loading the Model

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-1.5B-Instruct"
)

tokenizer = AutoTokenizer.from_pretrained(
    "Qwen/Qwen2.5-1.5B-Instruct"
)

model = PeftModel.from_pretrained(
    base_model,
    "igmoiiz/math-tutor-qwen-lora"
)

---

Example

Input

John has 12 apples. He gives 5 to his friend and buys 8 more. How many apples does he have now?

Output

John starts with 12 apples.

After giving away 5:
12 − 5 = 7

After buying 8 more:
7 + 8 = 15

Final Answer: 15 apples.

---

Citation

If you use this model, please cite the original GSM8K paper:

@article{cobbe2021gsm8k,
  title={Training Verifiers to Solve Math Word Problems},
  author={Cobbe, Karl and others},
  journal={arXiv preprint arXiv:2110.14168},
  year={2021}
}

---

Author

Moiz Baloch

Computer Science Undergraduate

Machine Learning & Deep Learning Engineer

GitHub: https://github.com/igmoiiz LinkedIn: https://linkedin.com/in/moizbaloch