camembert_ccnet_classification_tools_qlora-8bit_fr

AntoineD/camembert_ccnet_classification_tools_qlora-8bit_fr

camembert_ccnet_classification_tools_qlora-8bit_fr is a machine learning model from AntoineD. Use it for the machine learning task on the model card, and read the license before you ship it in a product.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

Access

Public

Updated Nov 15, 2023

Repo size

623 MB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin5.2 MB · 62%

At a glance

Access: Public
Created: Nov 15, 2023
Updated: Nov 15, 2023
SHA: 75abd08f

Base models

Model card

From the Hugging Face model README

camembert_ccnet_classification_tools_qlora-8bit_fr

This model is a fine-tuned version of camembert/camembert-base-ccnet on the None dataset. It achieves the following results on the evaluation set:

Loss: 1.5066
Accuracy: 0.85
Learning Rate: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 24
eval_batch_size: 192
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Rate
2.0629	1.0	7	2.0715	0.125	0.0001
2.0638	2.0	14	2.0676	0.15	0.0001
2.0614	3.0	21	2.0661	0.1	0.0001
2.0676	4.0	28	2.0622	0.125	0.0001
2.0502	5.0	35	2.0530	0.15	0.0001
2.0409	6.0	42	2.0480	0.175	9e-05
2.0457	7.0	49	2.0431	0.1	0.0001
2.0401	8.0	56	2.0398	0.125	0.0001
2.0239	9.0	63	2.0283	0.275	0.0001
2.0257	10.0	70	2.0131	0.3	0.0001
1.999	11.0	77	1.9926	0.425	0.0001
2.0053	12.0	84	1.9866	0.4	8e-05
1.9893	13.0	91	1.9710	0.475	0.0001
1.9762	14.0	98	1.9572	0.475	0.0001
1.9726	15.0	105	1.9398	0.425	0.0001
1.9456	16.0	112	1.9152	0.5	0.0001
1.9425	17.0	119	1.8971	0.575	0.0001
1.9078	18.0	126	1.8788	0.625	7e-05
1.901	19.0	133	1.8608	0.65	0.0001
1.8865	20.0	140	1.8407	0.625	0.0001
1.8696	21.0	147	1.8234	0.675	0.0001
1.8618	22.0	154	1.7964	0.725	0.0001
1.8238	23.0	161	1.7778	0.725	0.0001
1.8164	24.0	168	1.7739	0.675	6e-05
1.823	25.0	175	1.7442	0.675	0.0001
1.7892	26.0	182	1.7265	0.7	0.0001
1.7865	27.0	189	1.7110	0.75	0.0001
1.7643	28.0	196	1.7114	0.725	0.0001
1.7613	29.0	203	1.6859	0.775	0.0001
1.7681	30.0	210	1.6686	0.775	5e-05
1.7251	31.0	217	1.6547	0.8	0.0000
1.7215	32.0	224	1.6431	0.8	0.0000
1.7304	33.0	231	1.6285	0.75	0.0000
1.7182	34.0	238	1.6164	0.8	0.0000
1.7099	35.0	245	1.6066	0.825	0.0000
1.6902	36.0	252	1.6060	0.8	4e-05
1.6839	37.0	259	1.5949	0.875	0.0000
1.6627	38.0	266	1.5822	0.875	0.0000
1.6558	39.0	273	1.5723	0.875	0.0000
1.6667	40.0	280	1.5639	0.85	0.0000
1.6663	41.0	287	1.5576	0.85	0.0000
1.642	42.0	294	1.5506	0.825	3e-05
1.6479	43.0	301	1.5471	0.825	0.0000
1.6425	44.0	308	1.5414	0.825	0.0000
1.6214	45.0	315	1.5351	0.85	0.0000
1.6331	46.0	322	1.5310	0.875	0.0000
1.6406	47.0	329	1.5281	0.875	0.0000
1.6332	48.0	336	1.5265	0.875	2e-05
1.6474	49.0	343	1.5218	0.875	0.0000
1.6122	50.0	350	1.5184	0.875	0.0000
1.6166	51.0	357	1.5162	0.875	0.0000
1.6245	52.0	364	1.5343	0.875	0.0000
1.6027	53.0	371	1.5272	0.85	0.0000
1.6305	54.0	378	1.5261	0.85	1e-05
1.6172	55.0	385	1.5173	0.85	0.0000
1.6109	56.0	392	1.5168	0.875	0.0000
1.6037	57.0	399	1.5154	0.875	5e-06
1.6031	58.0	406	1.4970	0.875	0.0000
1.617	59.0	413	1.4961	0.875	0.0000
1.5955	60.0	420	1.5066	0.85	0.0

Framework versions

Transformers 4.34.0
Pytorch 2.0.1+cu117
Datasets 2.14.5
Tokenizers 0.14.1

Created: Nov 15, 2023
Updated: Nov 15, 2023