lemexp-task1-v2-template_small_nodefs-deepseek-coder-1.3b-base-8lr-48epochs-eos-train-embeddings

yalhessi/lemexp-task1-v2-template_small_nodefs-deepseek-coder-1.3b-base-8lr-48epochs-eos-train-embeddings

lemexp-task1-v2-template_small_nodefs-deepseek-coder-1.3b-base-8lr-48epochs-eos-train-embeddings is a machine learning model from yalhessi. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as other.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

44% of all-time downloads

All-time downloads

187

Public

Repo size

14.1 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors13.3 GB · 94%

At a glance

Library: peft
License: other
Access: Public
Created: Oct 11, 2025
Updated: Oct 12, 2025
SHA: fe86f0ab

Base models

deepseek-ai/deepseek-coder-1.3b-base

Model card

From the Hugging Face model README

lemexp-task1-v2-template_small_nodefs-deepseek-coder-1.3b-base-8lr-48epochs-eos-train-embeddings

This model is a fine-tuned version of deepseek-ai/deepseek-coder-1.3b-base on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.1562

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0008
train_batch_size: 2
eval_batch_size: 2
seed: 42
distributed_type: multi-GPU
num_devices: 8
total_train_batch_size: 16
total_eval_batch_size: 16
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 48
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.2042	0.8002	2880	0.2019
0.1794	1.6004	5760	0.1920
0.1656	2.4007	8640	0.1802
0.156	3.2009	11520	0.1784
0.1557	4.0011	14400	0.1697
0.1503	4.8013	17280	0.1652
0.1451	5.6016	20160	0.1686
0.1409	6.4018	23040	0.1572
0.1342	7.2020	25920	0.1585
0.1381	8.0022	28800	0.1567
0.1312	8.8024	31680	0.1537
0.1298	9.6027	34560	0.1464
0.1216	10.4029	37440	0.1445
0.1175	11.2031	40320	0.1511
0.122	12.0033	43200	0.1385
0.1192	12.8036	46080	0.1401
0.1136	13.6038	48960	0.1440
0.1108	14.4040	51840	0.1391
0.1029	15.2042	54720	0.1392
0.1097	16.0044	57600	0.1359
0.1056	16.8047	60480	0.1310
0.0989	17.6049	63360	0.1366
0.0944	18.4051	66240	0.1309
0.09	19.2053	69120	0.1270
0.0953	20.0056	72000	0.1267
0.093	20.8058	74880	0.1223
0.0874	21.6060	77760	0.1249
0.0822	22.4062	80640	0.1270
0.0778	23.2064	83520	0.1290
0.0842	24.0067	86400	0.1277
0.0801	24.8069	89280	0.1188
0.0747	25.6071	92160	0.1199
0.073	26.4073	95040	0.1218
0.0668	27.2076	97920	0.1241
0.07	28.0078	100800	0.1236
0.0668	28.8080	103680	0.1221
0.0622	29.6082	106560	0.1198
0.058	30.4084	109440	0.1222
0.0532	31.2087	112320	0.1213
0.0581	32.0089	115200	0.1219
0.0546	32.8091	118080	0.1174
0.0506	33.6093	120960	0.1235
0.0471	34.4096	123840	0.1226
0.0414	35.2098	126720	0.1249
0.046	36.0100	129600	0.1266
0.0428	36.8102	132480	0.1221
0.0389	37.6104	135360	0.1213
0.0352	38.4107	138240	0.1275
0.032	39.2109	141120	0.1337
0.0346	40.0111	144000	0.1354
0.0322	40.8113	146880	0.1332
0.0303	41.6116	149760	0.1350
0.0263	42.4118	152640	0.1403
0.0237	43.2120	155520	0.1437
0.0261	44.0122	158400	0.1447
0.024	44.8124	161280	0.1423
0.0209	45.6127	164160	0.1464
0.0194	46.4129	167040	0.1517
0.0191	47.2131	169920	0.1562

Framework versions

PEFT 0.14.0
Transformers 4.47.0
Pytorch 2.5.1+cu124
Datasets 3.2.0
Tokenizers 0.21.0

Library: peft
License: other
Created: Oct 11, 2025
Updated: Oct 12, 2025