lemexp-task1-v2-template_small_notypes-deepseek-coder-1.3b-base-ddp-8lr-v2

yalhessi/lemexp-task1-v2-template_small_notypes-deepseek-coder-1.3b-base-ddp-8lr-v2

lemexp-task1-v2-template_small_notypes-deepseek-coder-1.3b-base-ddp-8lr-v2 is a machine learning model from yalhessi. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as other.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

3% of all-time downloads

All-time downloads

393

Public

Repo size

1.3 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors269 MB · 100%

At a glance

Library: peft
License: other
Access: Public
Created: Apr 29, 2025
Updated: May 28, 2025
SHA: 2aad502c

Base models

deepseek-ai/deepseek-coder-1.3b-base

Model card

From the Hugging Face model README

lemexp-task1-v2-template_small_notypes-deepseek-coder-1.3b-base-ddp-8lr-v2

This model is a fine-tuned version of deepseek-ai/deepseek-coder-1.3b-base on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.1711

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0008
train_batch_size: 2
eval_batch_size: 2
seed: 42
distributed_type: multi-GPU
num_devices: 8
total_train_batch_size: 16
total_eval_batch_size: 16
optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
num_epochs: 12
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.4293	0.2001	720	0.3388
0.324	0.4001	1440	0.2977
0.282	0.6002	2160	0.2814
0.2707	0.8002	2880	0.2713
0.2595	1.0003	3600	0.2822
0.2434	1.2003	4320	0.2586
0.2403	1.4004	5040	0.2458
0.2355	1.6004	5760	0.2513
0.2342	1.8005	6480	0.2404
0.2284	2.0006	7200	0.2346
0.2176	2.2006	7920	0.2339
0.2148	2.4007	8640	0.2315
0.2127	2.6007	9360	0.2236
0.2108	2.8008	10080	0.2300
0.2124	3.0008	10800	0.2186
0.1962	3.2009	11520	0.2232
0.1993	3.4009	12240	0.2160
0.1944	3.6010	12960	0.2141
0.1945	3.8011	13680	0.2150
0.1934	4.0011	14400	0.2132
0.182	4.2012	15120	0.2050
0.1817	4.4012	15840	0.2079
0.1809	4.6013	16560	0.2013
0.1805	4.8013	17280	0.2045
0.1768	5.0014	18000	0.1979
0.1661	5.2014	18720	0.1919
0.1673	5.4015	19440	0.1962
0.1679	5.6016	20160	0.1925
0.168	5.8016	20880	0.1873
0.1623	6.0017	21600	0.1869
0.155	6.2017	22320	0.1875
0.1551	6.4018	23040	0.1869
0.1521	6.6018	23760	0.1870
0.1536	6.8019	24480	0.1816
0.1506	7.0019	25200	0.1825
0.1417	7.2020	25920	0.1867
0.1405	7.4021	26640	0.1795
0.1409	7.6021	27360	0.1808
0.1384	7.8022	28080	0.1754
0.1409	8.0022	28800	0.1767
0.1271	8.2023	29520	0.1753
0.1258	8.4023	30240	0.1742
0.1279	8.6024	30960	0.1737
0.126	8.8024	31680	0.1709
0.1255	9.0025	32400	0.1688
0.1138	9.2026	33120	0.1734
0.1134	9.4026	33840	0.1709
0.1149	9.6027	34560	0.1697
0.1143	9.8027	35280	0.1681
0.1106	10.0028	36000	0.1651
0.1011	10.2028	36720	0.1698
0.1004	10.4029	37440	0.1672
0.1003	10.6029	38160	0.1698
0.1004	10.8030	38880	0.1681
0.0999	11.0031	39600	0.1660
0.091	11.2031	40320	0.1721
0.0901	11.4032	41040	0.1714
0.0886	11.6032	41760	0.1719
0.0885	11.8033	42480	0.1711

Framework versions

PEFT 0.14.0
Transformers 4.47.0
Pytorch 2.5.1+cu124
Datasets 3.2.0
Tokenizers 0.21.0

Library: peft
License: other
Created: Apr 29, 2025
Updated: May 28, 2025