wav2vec2-base_than_I_did

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

wav2vec2-base_than_I_did

This model is a fine-tuned version of facebook/wav2vec2-base on the MatsRooth/than_I_did dataset. It achieves the following results on the evaluation set:

• Loss: 0.2077
• Accuracy: 0.9592

Model description

This is a binary classifier for the prosody of tokens of "I did". The label s is subject prominence. The label ns is the complement, with prominence either on "did" or afterwards.

Intended uses & limitations

Research on prosody.

Training and evaluation data

The utterances are collected on Youtube, aligned with the Youtube transcript using Kaldi, and cut to the words "I did" using Matlab. Labels were assigned by the experimenter, using 's' for tokens there the main clause subject differed from the than-clause subject, and 'ns' for other tokens. The labeling does not depend on prosody, though it correlates with it.

On the same problem using an SVM classifier, see Howell, Jonathan, Mats Rooth, and Michael Wagner, Acoustic classification of focus: On the web and in the lab (2016).

The class ns was reduced to 160 tokens, to match the number of tokens of s.

Training procedure

Training and evaluation use run_audio_classification.py from HuggingFace. The slurm script than_I_did.sub launches training.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 0
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20.0
• mixed_precision_training: Native AMP

Training results

Framework versions

• Transformers 4.36.0.dev0
• Pytorch 2.1.0+cu121
• Datasets 2.13.1
• Tokenizers 0.15.0