distilhubert-finetuned-gtzan

WasuratS/distilhubert-finetuned-gtzan

distilhubert-finetuned-gtzan is a audio classification model from WasuratS. Use it for the audio classification task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

2% of all-time downloads

All-time downloads

2.3K

Public

Repo size

17.8 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin94.8 MB · 99%

At a glance

Task: Audio Classification
Library: transformers
License: apache-2.0
Model type: hubert
Access: Public
Created: Jul 8, 2023
Updated: Jul 12, 2023
SHA: 2496c054

Datasets

marsyas/gtzan

Model card

From the Hugging Face model README

distilhubert-finetuned-gtzan

This model is a fine-tuned version of ntu-spml/distilhubert on the GTZAN dataset. It achieves the following results on the evaluation set on best epoch:

Loss: 0.7305
Accuracy: 0.9

Model description

Distilhubert is distilled version of the HuBERT and pretrained on data set with 16k frequency. <br/> Architecture of this model is CTC or Connectionist Temporal Classification is a technique that is used with encoder-only transformer. <br/>

Training and evaluation data

Training + Evaluation data set is GTZAN which is a popular dataset of 999 songs for music genre classification. <br/> Each song is a 30-second clip from one of 10 genres of music, spanning disco to metal.<br/> Train set is 899 songs and Evaluation set is 100 songs remainings.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 4
eval_batch_size: 4
seed: 42
distributed_type: multi-GPU
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 35
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
2.1728	1.0	225	2.0896	0.42
1.4211	2.0	450	1.4951	0.55
1.2155	3.0	675	1.0669	0.72
1.0175	4.0	900	0.8862	0.69
0.3516	5.0	1125	0.6265	0.83
0.6135	6.0	1350	0.6485	0.78
0.0807	7.0	1575	0.6567	0.78
0.0303	8.0	1800	0.7615	0.83
0.2663	9.0	2025	0.6612	0.86
0.0026	10.0	2250	0.8354	0.85
0.0337	11.0	2475	0.6768	0.87
0.0013	12.0	2700	0.7718	0.87
0.001	13.0	2925	0.7570	0.88
0.0008	14.0	3150	0.8170	0.89
0.0006	15.0	3375	0.7920	0.89
0.0005	16.0	3600	0.9859	0.83
0.0004	17.0	3825	0.8190	0.9
0.0003	18.0	4050	0.7305	0.9
0.0003	19.0	4275	0.8025	0.88
0.0002	20.0	4500	0.8208	0.87
0.0003	21.0	4725	0.7358	0.88
0.0002	22.0	4950	0.8681	0.87
0.0002	23.0	5175	0.7831	0.9
0.0003	24.0	5400	0.8583	0.88
0.0002	25.0	5625	0.8138	0.88
0.0002	26.0	5850	0.7871	0.89
0.0002	27.0	6075	0.8893	0.88
0.0002	28.0	6300	0.8284	0.89
0.0001	29.0	6525	0.8388	0.89
0.0001	30.0	6750	0.8305	0.9
0.0001	31.0	6975	0.8377	0.88
0.0153	32.0	7200	0.8496	0.88
0.0001	33.0	7425	0.8381	0.88
0.0001	34.0	7650	0.8440	0.88
0.0001	35.0	7875	0.8458	0.88

Framework versions

Transformers 4.29.2
Pytorch 1.13.1+cu117
Datasets 2.12.0
Tokenizers 0.13.3

Task: Audio Classification
Library: transformers
Type: hubert
License: apache-2.0
Created: Jul 8, 2023
Updated: Jul 12, 2023