wav2vec2-large-xls-r-300m-dm32

wav2vec2-large-xls-r-300m-dm32

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m for binary speech classification (dementia vs. no dementia). The objective is to detect cognitive impairment from spontaneous speech audio.

---

🧠 Model Description

The model leverages the multilingual XLS-R 300M wav2vec2 architecture and fine-tunes a classification head for dementia detection.

Key characteristics:

• Pretrained speech representations from wav2vec2
• Fine-tuned classification head
• Random audio chunk sampling during training
• Mixed precision training (Native AMP)
• Feature encoder frozen during early training for stability

The model accepts 16 kHz audio input and outputs probabilities for two classes:

• 0 → Dementia
• 1 → No Dementia

---

📂 Training and Evaluation Data

Dataset composition:

• 227 training samples
• 48 validation samples
• Two balanced class labels: dementia and nodementia

Audio preprocessing:

• Resampled to 16 kHz
• Random 32 second segments used during training
• Standardized waveform normalization

Due to the relatively small dataset size, some variance across training runs is expected.

---

⚙️ Training Procedure

Fine-tuning was conducted using the Hugging Face Trainer API.

Hyperparameters

• Learning rate: 1e-4
• Train batch size: 8
• Eval batch size: 8
• Gradient accumulation steps: 2
• Effective batch size: 16
• Number of epochs: 22
• Optimizer: AdamW (fused implementation)
• Betas: (0.9, 0.999)
• Epsilon: 1e-8
• LR scheduler: Linear
• Mixed precision: Native AMP
• Seed: 42

---

🖥️ Hardware

Training was performed on:

• NVIDIA A100 GPU
• CUDA-enabled PyTorch 2.10

---

📊 Evaluation Results (Validation Set)

Final validation performance after 22 epochs:

• Validation Loss: 0.5444
• Accuracy: 0.7778
• Precision: 0.6818
• Recall: 0.9375
• F1 Score: 0.7895

Confusion Matrix

[[13, 7], [ 1,15]]

Where:

• True Negatives: 13
• False Positives: 7
• False Negatives: 1
• True Positives: 15

---

📈 Training Progress Snapshot

---

🔎 Interpretation

The model demonstrates high recall (0.94), meaning it correctly identifies most dementia cases. There are moderate false positives, where some non-dementia samples are classified as dementia. Given the screening context, prioritizing recall is desirable to reduce missed positive cases.

---

🚨 Intended Use

This model is intended for:

• Academic research
• Proof-of-concept dementia screening experiments
• Speech-based cognitive impairment studies

This model is not intended for clinical diagnosis and should not be used for medical decision-making.

---

⚠️ Limitations

• Small dataset size may limit generalization.
• Performance may vary across accents, languages, and recording environments.
• Further validation on larger and more diverse datasets is required.

---

📚 Framework Versions

• Transformers 5.0.0
• PyTorch 2.10.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2