qwen2.5-7b-metamath-50k-lora-checkpoints

Qwen2.5-7B MetaMath 50K LoRA Checkpoints

Three epoch-level LoRA checkpoints from supervised fine-tuning of the Qwen2.5-7B base model on a seed-42 sample of 50,000 MetaMathQA examples.

Checkpoints

Each subfolder contains the PEFT adapter, tokenizer files, and the optimizer, scheduler, RNG, and Trainer states required to inspect or resume that training checkpoint.

Training configuration

• Base model: Qwen/Qwen2.5-7B (base, not instruct)
• Method: LoRA SFT
• LoRA rank / alpha / dropout: 16 / 32 / 0.05
• Target modules: all linear projection modules
• Training samples: 50,000
• Epochs: 3
• Maximum sequence length: 4096
• Effective global batch size: 32
• Learning rate: 2e-5, cosine schedule, 5% warmup
• Precision: BF16
• Chat template mode: non-thinking
• Seed: 42

Loading one checkpoint

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "Qwen/Qwen2.5-7B"
adapter_repo_id = "Han03430/qwen2.5-7b-metamath-50k-lora-checkpoints"

tokenizer = AutoTokenizer.from_pretrained(
    adapter_repo_id,
    subfolder="checkpoint-1563",
)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(
    base_model,
    adapter_repo_id,
    subfolder="checkpoint-1563",
)

Replace checkpoint-1563 with checkpoint-3126 or checkpoint-4689 to load the other epochs.

Only the original training checkpoints are included here. Evaluation outputs and post-hoc edited adapters are intentionally excluded.