Downloads · 30 days
0
Qwen2.6-14B-Math-LoRA
qingy2024/Qwen2.6-14B-Math-LoRA
Qwen2.6-14B-Math-LoRA is a machine learning model from qingy2024. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
- Developed by: qingy2024 - License: apache-2.0 - Finetuned from model : unsloth/qwen2.5-14b-bnb-4bit
Access
Public
Updated Dec 4, 2024
Repo size
287 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors275 MB · 95%
At a glance
- Library
- transformers
- License
- apache-2.0
- Access
- Public
- Created
- Dec 4, 2024
- Updated
- Dec 4, 2024
- SHA
- 17ff808a
Base models
Model card
From the Hugging Face model README
Uploaded model
- Developed by: qingy2024
- License: apache-2.0
- Finetuned from model : unsloth/qwen2.5-14b-bnb-4bit
Trained on qingy2024/metamathqa-30k for 500 steps.
| Parameter | Value | Description |
|---|---|---|
| per_device_train_batch_size | 4 | Number of samples per batch on each device during training. |
| gradient_accumulation_steps | 3 | Number of steps to accumulate gradients before updating. |
| warmup_steps | 5 | Number of steps for learning rate warmup. |
| max_steps | 500 | Total number of training steps. |
| learning_rate | 2e-4 | Initial learning rate for training. |
| logging_steps | 1 | Frequency of logging updates (in steps). |
| optim | adamw_8bit | Optimizer used for training. |
| weight_decay | 0.01 | Weight decay regularization coefficient. |
| lr_scheduler_type | cosine | Type of learning rate scheduler. |
| seed | 3407 | Random seed for reproducibility. |
| packing | True | Enables sequence packing for faster training. |
| max_seq_length | 2048 | Maximum token length per sequence. |
| dataset_num_proc` | 2 | Number of processes for dataset preparation. |
- Library
- transformers
- License
- apache-2.0
- Languages
- en
- Created
- Dec 4, 2024
- Updated
- Dec 4, 2024