model_broadclass_onSet2

JovialValley/model_broadclass_onSet2

model_broadclass_onSet2 is a automatic speech recognition model from JovialValley. Use it when you need speech turned into text. It is set up for transformers. The card lists the license as apache-2.0.

should probably proofread and complete it, then remove this comment. --

Downloads · 30 days

23% of all-time downloads

All-time downloads

Public

Repo size

6.3 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin1.3 GB · 100%

At a glance

Task: Automatic Speech Recognition
Library: transformers
License: apache-2.0
Model type: wav2vec2
Access: Public
Created: Dec 21, 2022
Updated: Dec 21, 2022
SHA: 8d78abcf

Model card

From the Hugging Face model README

model_broadclass_onSet2

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.5931
0 Precision: 1.0
0 Recall: 0.9615
0 F1-score: 0.9804
0 Support: 26
1 Precision: 0.9730
1 Recall: 0.9231
1 F1-score: 0.9474
1 Support: 39
2 Precision: 1.0
2 Recall: 1.0
2 F1-score: 1.0
2 Support: 19
3 Precision: 0.8125
3 Recall: 1.0
3 F1-score: 0.8966
3 Support: 13
Accuracy: 0.9588
Macro avg Precision: 0.9464
Macro avg Recall: 0.9712
Macro avg F1-score: 0.9561
Macro avg Support: 97
Weighted avg Precision: 0.9640
Weighted avg Recall: 0.9588
Weighted avg F1-score: 0.9597
Weighted avg Support: 97
Wer: 0.6924
Mtrix: [[0, 1, 2, 3], [0, 25, 1, 0, 0], [1, 0, 36, 0, 3], [2, 0, 0, 19, 0], [3, 0, 0, 0, 13]]

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 8
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 200
num_epochs: 80
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	0 Precision	0 Recall	0 F1-score	0 Support	1 Precision	1 Recall	1 F1-score	1 Support	2 Precision	2 Recall	2 F1-score	2 Support	3 Precision	3 Recall	3 F1-score	3 Support	Accuracy	Macro avg Precision	Macro avg Recall	Macro avg F1-score	Macro avg Support	Weighted avg Precision	Weighted avg Recall	Weighted avg F1-score	Weighted avg Support	Wer	Mtrix
2.3566	4.16	100	2.1836	0.2680	1.0	0.4228	26	0.0	0.0	0.0	39	0.0	0.0	0.0	19	0.0	0.0	0.0	13	0.2680	0.0670	0.25	0.1057	97	0.0718	0.2680	0.1133	97	0.9869	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 39, 0, 0, 0], [2, 19, 0, 0, 0], [3, 13, 0, 0, 0]]
2.2923	8.33	200	2.1159	0.2680	1.0	0.4228	26	0.0	0.0	0.0	39	0.0	0.0	0.0	19	0.0	0.0	0.0	13	0.2680	0.0670	0.25	0.1057	97	0.0718	0.2680	0.1133	97	0.9869	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 39, 0, 0, 0], [2, 19, 0, 0, 0], [3, 13, 0, 0, 0]]
1.9868	12.49	300	1.9923	0.2680	1.0	0.4228	26	0.0	0.0	0.0	39	0.0	0.0	0.0	19	0.0	0.0	0.0	13	0.2680	0.0670	0.25	0.1057	97	0.0718	0.2680	0.1133	97	0.9869	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 39, 0, 0, 0], [2, 19, 0, 0, 0], [3, 13, 0, 0, 0]]
1.7313	16.65	400	1.6081	0.2680	1.0	0.4228	26	0.0	0.0	0.0	39	0.0	0.0	0.0	19	0.0	0.0	0.0	13	0.2680	0.0670	0.25	0.1057	97	0.0718	0.2680	0.1133	97	0.9869	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 39, 0, 0, 0], [2, 19, 0, 0, 0], [3, 13, 0, 0, 0]]
1.6688	20.82	500	1.5971	0.2680	1.0	0.4228	26	0.0	0.0	0.0	39	0.0	0.0	0.0	19	0.0	0.0	0.0	13	0.2680	0.0670	0.25	0.1057	97	0.0718	0.2680	0.1133	97	0.9869	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 39, 0, 0, 0], [2, 19, 0, 0, 0], [3, 13, 0, 0, 0]]
1.5888	24.98	600	1.6098	0.2680	1.0	0.4228	26	0.0	0.0	0.0	39	0.0	0.0	0.0	19	0.0	0.0	0.0	13	0.2680	0.0670	0.25	0.1057	97	0.0718	0.2680	0.1133	97	0.9869	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 39, 0, 0, 0], [2, 19, 0, 0, 0], [3, 13, 0, 0, 0]]
1.5986	29.16	700	1.6984	0.2680	1.0	0.4228	26	0.0	0.0	0.0	39	0.0	0.0	0.0	19	0.0	0.0	0.0	13	0.2680	0.0670	0.25	0.1057	97	0.0718	0.2680	0.1133	97	0.9869	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 39, 0, 0, 0], [2, 19, 0, 0, 0], [3, 13, 0, 0, 0]]
1.5437	33.33	800	1.4933	0.2680	1.0	0.4228	26	0.0	0.0	0.0	39	0.0	0.0	0.0	19	0.0	0.0	0.0	13	0.2680	0.0670	0.25	0.1057	97	0.0718	0.2680	0.1133	97	0.9869	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 39, 0, 0, 0], [2, 19, 0, 0, 0], [3, 13, 0, 0, 0]]
1.1358	37.49	900	1.1118	0.2680	1.0	0.4228	26	0.0	0.0	0.0	39	0.0	0.0	0.0	19	0.0	0.0	0.0	13	0.2680	0.0670	0.25	0.1057	97	0.0718	0.2680	0.1133	97	0.9869	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 39, 0, 0, 0], [2, 19, 0, 0, 0], [3, 13, 0, 0, 0]]
0.983	41.65	1000	1.0538	0.3171	1.0	0.4815	26	1.0	0.0256	0.05	39	1.0	0.3158	0.4800	19	0.875	0.5385	0.6667	13	0.4124	0.7980	0.4700	0.4195	97	0.8002	0.4124	0.3325	97	0.9732	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 37, 1, 0, 1], [2, 13, 0, 6, 0], [3, 6, 0, 0, 7]]
0.96	45.82	1100	0.9324	0.4561	1.0	0.6265	26	1.0	0.3846	0.5556	39	1.0	0.6316	0.7742	19	1.0	1.0	1.0	13	0.6804	0.8640	0.7540	0.7391	97	0.8542	0.6804	0.6770	97	0.9510	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 24, 15, 0, 0], [2, 7, 0, 12, 0], [3, 0, 0, 0, 13]]
0.9569	49.98	1200	0.9106	0.52	1.0	0.6842	26	1.0	0.6410	0.7813	39	1.0	0.6316	0.7742	19	1.0	0.7692	0.8696	13	0.7526	0.88	0.7605	0.7773	97	0.8713	0.7526	0.7657	97	0.9343	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 14, 25, 0, 0], [2, 7, 0, 12, 0], [3, 3, 0, 0, 10]]
0.943	54.16	1300	0.9142	0.7879	1.0	0.8814	26	1.0	0.8205	0.9014	39	1.0	0.9474	0.9730	19	0.9286	1.0	0.9630	13	0.9175	0.9291	0.9420	0.9297	97	0.9336	0.9175	0.9183	97	0.9242	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 6, 32, 0, 1], [2, 1, 0, 18, 0], [3, 0, 0, 0, 13]]
0.9177	58.33	1400	0.9201	0.7879	1.0	0.8814	26	1.0	0.7692	0.8696	39	1.0	1.0	1.0	19	0.8667	1.0	0.9286	13	0.9072	0.9136	0.9423	0.9199	97	0.9253	0.9072	0.9062	97	0.9197	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 7, 30, 0, 2], [2, 0, 0, 19, 0], [3, 0, 0, 0, 13]]
0.873	62.49	1500	0.8556	0.8387	1.0	0.9123	26	1.0	0.8718	0.9315	39	1.0	0.9474	0.9730	19	0.9286	1.0	0.9630	13	0.9381	0.9418	0.9548	0.9449	97	0.9472	0.9381	0.9387	97	0.9293	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 4, 34, 0, 1], [2, 1, 0, 18, 0], [3, 0, 0, 0, 13]]
0.798	66.65	1600	0.8133	0.8966	1.0	0.9455	26	1.0	0.8974	0.9459	39	1.0	1.0	1.0	19	0.9286	1.0	0.9630	13	0.9588	0.9563	0.9744	0.9636	97	0.9627	0.9588	0.9587	97	0.9071	[[0, 1, 2, 3], [0, 26, 0, 0, 0], [1, 3, 35, 0, 1], [2, 0, 0, 19, 0], [3, 0, 0, 0, 13]]
0.7299	70.82	1700	0.7332	1.0	0.9615	0.9804	26	0.9744	0.9744	0.9744	39	1.0	1.0	1.0	19	0.9286	1.0	0.9630	13	0.9794	0.9757	0.9840	0.9794	97	0.9801	0.9794	0.9795	97	0.8636	[[0, 1, 2, 3], [0, 25, 1, 0, 0], [1, 0, 38, 0, 1], [2, 0, 0, 19, 0], [3, 0, 0, 0, 13]]
0.6432	74.98	1800	0.6808	1.0	0.9615	0.9804	26	0.9730	0.9231	0.9474	39	1.0	1.0	1.0	19	0.8125	1.0	0.8966	13	0.9588	0.9464	0.9712	0.9561	97	0.9640	0.9588	0.9597	97	0.7758	[[0, 1, 2, 3], [0, 25, 1, 0, 0], [1, 0, 36, 0, 3], [2, 0, 0, 19, 0], [3, 0, 0, 0, 13]]
0.6067	79.16	1900	0.5931	1.0	0.9615	0.9804	26	0.9730	0.9231	0.9474	39	1.0	1.0	1.0	19	0.8125	1.0	0.8966	13	0.9588	0.9464	0.9712	0.9561	97	0.9640	0.9588	0.9597	97	0.6924	[[0, 1, 2, 3], [0, 25, 1, 0, 0], [1, 0, 36, 0, 3], [2, 0, 0, 19, 0], [3, 0, 0, 0, 13]]

Framework versions

Transformers 4.25.1
Pytorch 1.13.0+cu116
Datasets 2.8.0
Tokenizers 0.13.2

Task: Automatic Speech Recognition
Library: transformers
Type: wav2vec2
License: apache-2.0
Created: Dec 21, 2022
Updated: Dec 21, 2022