qwen2.5-7b-sft-25-1

hZzy/qwen2.5-7b-sft-25-1

qwen2.5-7b-sft-25-1 is a machine learning model from hZzy. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

12% of all-time downloads

All-time downloads

Public

Repo size

2.2 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors2.2 GB · 99%

At a glance

Library: peft
License: apache-2.0
Model type: qwen2
Access: Public
Created: Mar 11, 2025
Updated: Mar 11, 2025
SHA: e0223d4f

Base models

Qwen/Qwen2.5-7B

Datasets

hZzy/SFT_new_full

Model card

From the Hugging Face model README

qwen2.5-7b-sft-25-1

This model is a fine-tuned version of Qwen/Qwen2.5-7B on the hZzy/SFT_new_full dataset. It achieves the following results on the evaluation set:

Loss: 2.5893

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 8
eval_batch_size: 8
seed: 42
distributed_type: multi-GPU
num_devices: 6
gradient_accumulation_steps: 8
total_train_batch_size: 384
total_eval_batch_size: 48
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.1
num_epochs: 15
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
2.9327	0.1449	5	2.8875
2.929	0.2899	10	2.8873
2.9354	0.4348	15	2.8865
2.9256	0.5797	20	2.8849
2.9257	0.7246	25	2.8832
2.9285	0.8696	30	2.8809
2.9109	1.0145	35	2.8775
2.9173	1.1594	40	2.8734
2.9079	1.3043	45	2.8687
2.903	1.4493	50	2.8624
2.9104	1.5942	55	2.8556
2.8864	1.7391	60	2.8483
2.892	1.8841	65	2.8409
2.8811	2.0290	70	2.8351
2.8712	2.1739	75	2.8280
2.8666	2.3188	80	2.8210
2.8675	2.4638	85	2.8146
2.8527	2.6087	90	2.8083
2.8424	2.7536	95	2.8023
2.8383	2.8986	100	2.7965
2.8324	3.0435	105	2.7907
2.835	3.1884	110	2.7850
2.8204	3.3333	115	2.7791
2.8159	3.4783	120	2.7729
2.7987	3.6232	125	2.7665
2.8087	3.7681	130	2.7599
2.7916	3.9130	135	2.7549
2.7854	4.0580	140	2.7479
2.7895	4.2029	145	2.7408
2.7705	4.3478	150	2.7342
2.764	4.4928	155	2.7296
2.7677	4.6377	160	2.7236
2.7496	4.7826	165	2.7178
2.7533	4.9275	170	2.7123
2.7335	5.0725	175	2.7073
2.7381	5.2174	180	2.7024
2.727	5.3623	185	2.6982
2.731	5.5072	190	2.6936
2.7255	5.6522	195	2.6886
2.7193	5.7971	200	2.6841
2.7148	5.9420	205	2.6795
2.7084	6.0870	210	2.6750
2.6999	6.2319	215	2.6707
2.7012	6.3768	220	2.6664
2.6906	6.5217	225	2.6622
2.6894	6.6667	230	2.6581
2.6894	6.8116	235	2.6543
2.6889	6.9565	240	2.6506
2.6799	7.1014	245	2.6472
2.6733	7.2464	250	2.6438
2.6702	7.3913	255	2.6406
2.6687	7.5362	260	2.6375
2.6652	7.6812	265	2.6345
2.6565	7.8261	270	2.6317
2.6633	7.9710	275	2.6292
2.6539	8.1159	280	2.6270
2.6578	8.2609	285	2.6248
2.6543	8.4058	290	2.6226
2.6483	8.5507	295	2.6204
2.6405	8.6957	300	2.6183
2.6428	8.8406	305	2.6165
2.64	8.9855	310	2.6147
2.6394	9.1304	315	2.6130
2.6354	9.2754	320	2.6115
2.6309	9.4203	325	2.6100
2.6346	9.5652	330	2.6086
2.6296	9.7101	335	2.6070
2.6342	9.8551	340	2.6055
2.6342	10.0	345	2.6042
2.6313	10.1449	350	2.6028
2.6267	10.2899	355	2.6017
2.6267	10.4348	360	2.6005
2.6219	10.5797	365	2.5993
2.6319	10.7246	370	2.5983
2.6107	10.8696	375	2.5974
2.6227	11.0145	380	2.5966
2.6186	11.1594	385	2.5958
2.6271	11.3043	390	2.5951
2.614	11.4493	395	2.5944
2.6075	11.5942	400	2.5937
2.6193	11.7391	405	2.5932
2.6239	11.8841	410	2.5927
2.6174	12.0290	415	2.5923
2.61	12.1739	420	2.5919
2.6158	12.3188	425	2.5915
2.6064	12.4638	430	2.5912
2.6202	12.6087	435	2.5909
2.6137	12.7536	440	2.5907
2.6201	12.8986	445	2.5904
2.6222	13.0435	450	2.5902
2.6195	13.1884	455	2.5900
2.6145	13.3333	460	2.5899
2.6088	13.4783	465	2.5897
2.6122	13.6232	470	2.5896
2.6127	13.7681	475	2.5896
2.6128	13.9130	480	2.5895
2.6088	14.0580	485	2.5894
2.6133	14.2029	490	2.5894
2.6093	14.3478	495	2.5894
2.6147	14.4928	500	2.5894
2.6176	14.6377	505	2.5894
2.6095	14.7826	510	2.5893

Framework versions

PEFT 0.11.1
Transformers 4.42.0
Pytorch 2.3.0+cu121
Datasets 3.2.0
Tokenizers 0.19.1

Library: peft
Type: qwen2
License: apache-2.0
Created: Mar 11, 2025
Updated: Mar 11, 2025