math-tutor-gemma-lora

Math Tutor LoRA Model (Gemma-2B)

Overview

This project presents a fine-tuned AI Tutor built on top of Gemma-2B using QLoRA (LoRA + 4-bit quantization). The model is designed to solve linear algebra equations step-by-step with structured and consistent outputs.

The goal of this project is to transform a general-purpose language model into a domain-specific tutoring system that demonstrates improved correctness and instructional clarity.

---

Model Details

• Base Model: google/gemma-2b
• Fine-Tuning Method: QLoRA (PEFT + bitsandbytes)
• Frameworks: Transformers, TRL, PEFT
• Task: Algebra equation solving (Linear equations)

---

Capabilities

The fine-tuned model is designed as a math tutor that:

• Solves equations step-by-step
• Produces structured outputs
• Provides consistent final answers
• Reduces incorrect reasoning compared to base model

---

Performance

Evaluation was conducted on 30 unseen test problems.

The fine-tuned model shows significant improvement in correctness and reliability.

---

Example

Input: Solve 2x + 4 = 10

Output: 2x = 10 - 4 2x = 6 x = 3

---

Training Details

• Dataset Type: Synthetic (programmatically generated)
• Dataset Size: ~600 samples
• Format: Instruction → Response
• Epochs: 3
• Batch Size: 2
• Quantization: 4-bit (nf4)
• Environment: Google Colab (T4 GPU)

---

How to Use

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

model_name = "google/gemma-2b"

tokenizer = AutoTokenizer.from_pretrained(model_name)

base_model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto"
)

model = PeftModel.from_pretrained(base_model, "your-username/math-tutor-gemma-lora")

prompt = """You are a math tutor.
Solve step-by-step.

### Question:
Solve 2x + 4 = 10

### Answer:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=100)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Future Improvements

• Expand dataset to include advanced algebra and word problems
• Improve pedagogical reasoning quality
• Add adaptive tutoring (interactive feedback)
• Deploy as API / web interface

---

Repository Structure

• lora-adapter/ → Fine-tuned LoRA weights
• README.md → Project documentation

---

Author

Mohamed Fasidh

[email protected]

🔗 https://github.com/Mohamed-Fasidh

🔗 https://www.linkedin.com/in/mohamed-fasidh/

---

License

This project uses publicly available models and synthetic data. Ensure compliance with base model licensing when using commercially.