Downloads · 30 days
30
45% of all-time downloads
Llama-3.1-8B-Instruct-MetaMathQA-50K-LoRA
tianzl66/Llama-3.1-8B-Instruct-MetaMathQA-50K-LoRA
Llama-3.1-8B-Instruct-MetaMathQA-50K-LoRA is a machine learning model from tianzl66. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.
This repository contains the vanilla LoRA adapter obtained by fine-tuning Llama-3.1-8B-Instruct on 50K MetaMathQA examples.
All-time downloads
67
Public
Repo size
168 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors168 MB · 100%
At a glance
- Library
- peft
- Access
- Public
- Created
- Aug 27, 2026
- Updated
- Aug 27, 2026
- SHA
- fc73b642
Base models
Model card
From the Hugging Face model README
Llama-3.1-8B-Instruct + MetaMathQA-50K LoRA
This repository contains the vanilla LoRA adapter obtained by fine-tuning Llama-3.1-8B-Instruct on 50K MetaMathQA examples.
This checkpoint serves as the LoRA baseline for the Spectral Surgery math experiments.
Base Model
meta-llama/Llama-3.1-8B-Instruct
Training
- Dataset: MetaMathQA
- Samples: 50K
- LoRA rank: 16
Evaluation
Evaluation on GSM8K.
| Model | GSM8K |
|---|---|
| Base | 65.20% (860/1319) |
| MetaMathQA-50K LoRA | 77.18% (1018/1319) |
| HNS 8+2, o_proj + down_proj | 78.39% (1034/1319) |
| HNS 8+2, all modules | 79.38% (1047/1319) |
| HNS 4+1, o_proj + down_proj | 78.17% (1031/1319) |
| HNS 4+1, all modules | 79.38% (1047/1319) |
The LoRA checkpoint improves GSM8K accuracy by 11.98 percentage points over the base model.
- Library
- peft
- Created
- Aug 27, 2026
- Updated
- Aug 27, 2026