wav2vec2-xls-r-300m-sr-cv8

comodoro/wav2vec2-xls-r-300m-sr-cv8

wav2vec2-xls-r-300m-sr-cv8 is a automatic speech recognition model from comodoro. Use it when you need speech turned into text. It is set up for transformers. The card lists the license as apache-2.0.

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the commonvoice dataset. It achieves the following results on the evaluation set: - Loss: 1.7302 - Wer: 0.4825 - Cer: 0.1847

Downloads · 30 days

2% of all-time downloads

All-time downloads

829

Public

Repo size

2.5 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.bin1.3 GB · 100%

At a glance

Task: Automatic Speech Recognition
Library: transformers
License: apache-2.0
Model type: wav2vec2
Access: Public
Created: Mar 2, 2022
Updated: Mar 24, 2022
SHA: 75ce1f7e

Model card

From the Hugging Face model README

Serbian wav2vec2-xls-r-300m-sr-cv8

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the common_voice dataset. It achieves the following results on the evaluation set:

Loss: 1.7302
Wer: 0.4825
Cer: 0.1847

Evaluation on mozilla-foundation/common_voice_8_0 gave the following results:

WER: 0.48530097993467103
CER: 0.18413288165227845

Evaluation on speech-recognition-community-v2/dev_data gave the following results:

WER: 0.9718373107518604
CER: 0.8302740620263108

The model can be evaluated using the attached eval.py script:

python eval.py --model_id comodoro/wav2vec2-xls-r-300m-sr-cv8 --dataset mozilla-foundation/common-voice_8_0 --split test --config sr

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 16
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 300
num_epochs: 800
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
5.6536	15.0	1200	2.9744	1.0	1.0
2.7935	30.0	2400	1.6613	0.8998	0.4670
1.6538	45.0	3600	0.9248	0.6918	0.2699
1.2446	60.0	4800	0.9151	0.6452	0.2398
1.0766	75.0	6000	0.9110	0.5995	0.2207
0.9548	90.0	7200	1.0273	0.5921	0.2149
0.8919	105.0	8400	0.9929	0.5646	0.2117
0.8185	120.0	9600	1.0850	0.5483	0.2069
0.7692	135.0	10800	1.1001	0.5394	0.2055
0.7249	150.0	12000	1.1018	0.5380	0.1958
0.6786	165.0	13200	1.1344	0.5114	0.1941
0.6432	180.0	14400	1.1516	0.5054	0.1905
0.6009	195.0	15600	1.3149	0.5324	0.1991
0.5773	210.0	16800	1.2468	0.5124	0.1903
0.559	225.0	18000	1.2186	0.4956	0.1922
0.5298	240.0	19200	1.4483	0.5333	0.2085
0.5136	255.0	20400	1.2871	0.4802	0.1846
0.4824	270.0	21600	1.2891	0.4974	0.1885
0.4669	285.0	22800	1.3283	0.4942	0.1878
0.4511	300.0	24000	1.4502	0.5002	0.1994
0.4337	315.0	25200	1.4714	0.5035	0.1911
0.4221	330.0	26400	1.4971	0.5124	0.1962
0.3994	345.0	27600	1.4473	0.5007	0.1920
0.3892	360.0	28800	1.3904	0.4937	0.1887
0.373	375.0	30000	1.4971	0.4946	0.1902
0.3657	390.0	31200	1.4208	0.4900	0.1821
0.3559	405.0	32400	1.4648	0.4895	0.1835
0.3476	420.0	33600	1.4848	0.4946	0.1829
0.3276	435.0	34800	1.5597	0.4979	0.1873
0.3193	450.0	36000	1.7329	0.5040	0.1980
0.3078	465.0	37200	1.6379	0.4937	0.1882
0.3058	480.0	38400	1.5878	0.4942	0.1921
0.2987	495.0	39600	1.5590	0.4811	0.1846
0.2931	510.0	40800	1.6001	0.4825	0.1849
0.276	525.0	42000	1.7388	0.4942	0.1918
0.2702	540.0	43200	1.7037	0.4839	0.1866
0.2619	555.0	44400	1.6704	0.4755	0.1840
0.262	570.0	45600	1.6042	0.4751	0.1865
0.2528	585.0	46800	1.6402	0.4821	0.1865
0.2442	600.0	48000	1.6693	0.4886	0.1862
0.244	615.0	49200	1.6203	0.4765	0.1792
0.2388	630.0	50400	1.6829	0.4830	0.1828
0.2362	645.0	51600	1.8100	0.4928	0.1888
0.2224	660.0	52800	1.7746	0.4932	0.1899
0.2218	675.0	54000	1.7752	0.4946	0.1901
0.2201	690.0	55200	1.6775	0.4788	0.1844
0.2147	705.0	56400	1.7085	0.4844	0.1851
0.2103	720.0	57600	1.7624	0.4848	0.1864
0.2101	735.0	58800	1.7213	0.4783	0.1835
0.1983	750.0	60000	1.7452	0.4848	0.1856
0.2015	765.0	61200	1.7525	0.4872	0.1869
0.1969	780.0	62400	1.7443	0.4844	0.1852
0.2043	795.0	63600	1.7302	0.4825	0.1847

Framework versions

Transformers 4.16.2
Pytorch 1.10.1+cu102
Datasets 1.18.3
Tokenizers 0.11.0

Task: Automatic Speech Recognition
Library: transformers
Type: wav2vec2
License: apache-2.0
Languages: sr
Created: Mar 2, 2022
Updated: Mar 24, 2022