Downloads · 30 days
0
Qwen2.5-14B-Math-LoRA-Iter2
qingy2024/Qwen2.5-14B-Math-LoRA-Iter2
Qwen2.5-14B-Math-LoRA-Iter2 is a machine learning model from qingy2024. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
- Developed by: qingy2019 - License: apache-2.0 - Finetuned from model : unsloth/qwen2.5-14b-bnb-4bit
Access
Public
Updated Dec 3, 2024
Repo size
562 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors551 MB · 97%
At a glance
- Library
- transformers
- License
- apache-2.0
- Access
- Public
- Created
- Dec 3, 2024
- Updated
- Dec 3, 2024
- SHA
- d6c8138a
Base models
Model card
From the Hugging Face model README
Uploaded model
- Developed by: qingy2019
- License: apache-2.0
- Finetuned from model : unsloth/qwen2.5-14b-bnb-4bit
Huge thanks to Unsloth and the Huggingface TRL library.
This model is Qwen 2.5 14B fine tuned for a full epoch on the high quality garage-bAInd/Open-Platypus dataset for STEM reasoning.
| Training Detail | Value |
|---|---|
| Epochs | 1 |
| Steps | 2077 |
| Loss | 0.4218 |
| Batch size | 4 |
| Gradient Acc. Steps | 3 |
| Learning Rate | 2e-4 |
| LR Scheduler | cosine |
| Rank | 32 |
| Rank-Stabilized LoRA | Yes |
| Warm up steps | 5 |
| Weight Decay | 0.01 |
| Seed | 3407 |
- Library
- transformers
- License
- apache-2.0
- Languages
- en
- Created
- Dec 3, 2024
- Updated
- Dec 3, 2024