lfm2-math-finetuned

LFM2-350M-Math - Fine-tuned

Model Description

This is a LoRA fine-tuned version of the LiquidAI/LFM2-350M-Math model on the HuggingFaceH4/MATH-500 dataset.
The model is designed to answer math questions and generate step-by-step solutions in natural language.

---

Intended Uses

• Solve math problems directly in natural language.
• Serve as a base for further fine-tuning on other math datasets.
• Educational tools, tutoring systems, or research in automated math reasoning.

---

Out-of-Scope Uses

• Not intended for general reasoning beyond mathematics.
• May fail or hallucinate on complex, unseen problem types.
• Should not be used for critical calculations without verification.

---

Limitations & Biases

• The training dataset is small (500 problems), so the model may overfit.
• Step-by-step reasoning is learned from dataset patterns, not true reasoning.
• Accuracy can vary; always verify outputs for correctness.

---

Training Details

• Base Model: LiquidAI/LFM2-350M-Math
• Dataset: HuggingFaceH4/MATH-500
• Fine-tuning libraries: transformers, datasets, peft, accelerate, bitsandbytes
• Training arguments:
  • num_train_epochs: 3
  • per_device_train_batch_size: 4
  • gradient_accumulation_steps: 4
  • learning_rate: 2e-4
  • fp16: True

---

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "LiquidAI/LFM2-350M-Math"
adapter_repo = "Nrex/lfm2-math-finetuned"

tokenizer = AutoTokenizer.from_pretrained(adapter_repo)
model = AutoModelForCausalLM.from_pretrained(base_model)
model = PeftModel.from_pretrained(model, adapter_repo)

inputs = tokenizer("What is 12*13?", return_tensors="pt")
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) 

---

Evaluation

• Tested on a held-out 10% split of the dataset.
• Small dataset; outputs should be verified for correctness.
• Accuracy is limited by the dataset size