hubert_large-german-IPA-v2

---

HuBERT Large German IPA Phoneme Scorer (v2)

---

Key Highlights & Performance

• Comprehensive Multi-Domain Training: Scaled from clean audiobooks to diverse acoustic environments including parliamentary debates, crowd-sourced phone audio, and studio recordings.
• Superior Accuracy: Reached an Evaluation Phoneme Error Rate (PER) of 22.28% (Accuracy > 77.7%) on unseen validation data, outperforming v1 across all acoustic categories.
• Low Latency: Average inference latency of ~32 ms per utterance on NVIDIA L4 (Tensor Cores / FP16), enabling real-time scoring in production.

---

📚 Training Dataset Architecture (123 GB Mega Dataset)

The model was fine-tuned on a composite dataset q1805/german-pronuncheck-mega-dataset, combining three complementary sources:

• 1.German Parliamentary Debates (Bundestag Corpus): Fast, spontaneous political discourse with natural room acoustics and public address microphones.
• 2.Mozilla Common Voice (German v17): Thousands of diverse speakers recorded on consumer smartphones and PC headsets with varied regional German dialects.
• 3.Multilingual LibriSpeech (MLS German): High-fidelity studio audiobook narrations.
• Total Samples: 1,249,116 training utterances + 86,575 validation utterances.
• Phonemizer: Converted to German IPA via espeak-ng using a fork-safe parallel pipeline (preserve_punctuation=False, with_stress=True).

---

⚙️ System-Level Engineering & Training Parameters

• Compute Infrastructure: Google Cloud Platform (GCP) Compute Engine VM.
• Accelerator: 1x NVIDIA L4 Tensor Core GPU (24 GB GDDR6 VRAM, Ada Lovelace architecture).
• Host Resources: 4 vCPUs, 16 GB RAM, 500 GB High-Throughput NVMe SSD.
• Total Training Time: 70 hours 25 minutes (73,050 total steps, 3 full epochs).

🛠️ Hardware & Memory Optimization Techniques:

• Mixed Precision (fp16=True): Enabled NVIDIA L4 Tensor Cores, cutting memory by 50% and doubling matrix throughput.
• Gradient Checkpointing (gradient_checkpointing=True): Mitigated memory spikes from long audio sequences ($O(N^2)$ attention matrices).
• Multi-Worker I/O (dataloader_num_workers=4): Eliminated GPU starvation by asynchronously feeding audio arrays.
• Effective Batch Size = 32: Configured via per_device_train_batch_size = 2 and gradient_accumulation_steps = 16.
• Memory Defragmentation: PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True.
• Learning Rate Policy: Linear warmup (1,000 steps) to 3e-5, followed by linear decay to 0.0.

---

📈 Comprehensive Benchmark Comparisons

1. Out-of-Domain Benchmark: Mozilla Common Voice Spontaneous Speech 4.0 (sps-corpus-4.0)

Evaluated on 100% of the completely unseen Mozilla Common Voice Spontaneous Speech 4.0 German dataset (natural speech with diverse accents and background noise):

---

2. Multi-Domain Benchmark: Mega Dataset 10% Held-Out Test Split (86,575 Utterances from q1805/german-pronuncheck-mega-dataset)

Evaluated on the independent 10% test split extracted from mega_dataset (comprising 86,575 utterances across Parliamentary debates, crowdsourced Mozilla audio, and MLS studio audio that neither model touched during training):

---

💻 Quickstart Inference Code

import torch
import librosa
from transformers import Wav2Vec2Processor, HubertForCTC
# Load production model and processor
REPO_ID = "q1805/hubert-german-IPA-large-v2"
processor = Wav2Vec2Processor.from_pretrained(REPO_ID)
model = HubertForCTC.from_pretrained(REPO_ID).eval()
# Load 16kHz audio
audio_path = "german_speech_sample.wav"
audio, sr = librosa.load(audio_path, sr=16000)
# Forward pass
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    logits = model(inputs.input_values).logits
    predicted_ids = torch.argmax(logits, dim=-1)
    ipa_transcription = processor.batch_decode(predicted_ids)[0]
print("Predicted German IPA:", ipa_transcription)