sparql-qwen

PopularPenguin/sparql-qwen

sparql-qwen is a machine learning model from PopularPenguin. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

41% of all-time downloads

All-time downloads

Public

Repo size

924 KB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.0914 KB · 99%

At a glance

Library: peft
License: apache-2.0
Access: Public
Created: Dec 16, 2024
Updated: Dec 17, 2024
SHA: 0b94368e

Base models

Qwen/Qwen2.5-Coder-0.5B-Instruct

Datasets

arrow

Model card

From the Hugging Face model README

sparql-qwen

This model is a fine-tuned version of Qwen/Qwen2.5-Coder-0.5B-Instruct on the arrow dataset. It achieves the following results on the evaluation set:

Loss: 0.5436

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 100
num_epochs: 10
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.7618	0.1048	500	0.7231
0.7397	0.2096	1000	0.6676
0.7213	0.3143	1500	0.6440
0.7047	0.4191	2000	0.6283
0.6905	0.5239	2500	0.6181
0.6822	0.6287	3000	0.6081
0.6651	0.7334	3500	0.6007
0.662	0.8382	4000	0.5938
0.6535	0.9430	4500	0.5889
0.562	1.0478	5000	0.5846
0.4974	1.1526	5500	0.5820
0.5317	1.2573	6000	0.5778
0.572	1.3621	6500	0.5743
0.5167	1.4669	7000	0.5718
0.5479	1.5717	7500	0.5692
0.5368	1.6764	8000	0.5659
0.5622	1.7812	8500	0.5643
0.5146	1.8860	9000	0.5621
0.509	1.9908	9500	0.5602
0.5536	2.0956	10000	0.5589
0.5035	2.2003	10500	0.5592
0.5399	2.3051	11000	0.5567
0.5247	2.4099	11500	0.5553
0.5365	2.5147	12000	0.5549
0.4425	2.6194	12500	0.5545
0.4761	2.7242	13000	0.5524
0.5368	2.8290	13500	0.5509
0.5214	2.9338	14000	0.5494
0.519	3.0386	14500	0.5496
0.5606	3.1433	15000	0.5492
0.5362	3.2481	15500	0.5476
0.5275	3.3529	16000	0.5476
0.5159	3.4577	16500	0.5464
0.5171	3.5624	17000	0.5461
0.5242	3.6672	17500	0.5454
0.5053	3.7720	18000	0.5445
0.512	3.8768	18500	0.5441
0.5259	3.9816	19000	0.5428
0.4363	4.0863	19500	0.5437
0.4784	4.1911	20000	0.5440
0.4703	4.2959	20500	0.5448
0.4467	4.4007	21000	0.5436

Framework versions

PEFT 0.14.0
Transformers 4.46.3
Pytorch 2.4.0
Datasets 3.1.0
Tokenizers 0.20.3

Library: peft
License: apache-2.0
Created: Dec 16, 2024
Updated: Dec 17, 2024