hubert-base-phoneme-en

hubert-base-phoneme-en

<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>

This model is a fine-tuned version of facebook/hubert-base-ls960 for phoneme-level CTC prediction with the 41-token ARPABET vocabulary used by peacock-asr.

It is a P003 backbone artifact, not the final pronunciation-scoring result. The downstream GOP/GOPT evaluation on SpeechOcean762 happens in the separate P003 eval sweeps.

Model description

• Base model: HuBERT-base (facebook/hubert-base-ls960, 95M parameters)
• Fine-tuning objective: phoneme-level CTC
• Training data: LibriSpeech 960h
• Output vocabulary: repo-standard ARPABET token set used for pronunciation scoring backends
• Hugging Face backend string for this repo: hf:Peacockery/hubert-base-phoneme-en

This artifact is intended to serve as the phoneme-posterior generator for the P003 compact-backbone comparison:

• wav2vec2-base (95M) vs
• HuBERT-base (95M) vs
• w2v-bert-2.0 (600M) vs
• future smaller backbones such as Citrinet

Intended uses & limitations

Intended use:

• research on phoneme posterior extraction
• backend for GOP-SF feature extraction
• controlled backbone swaps inside the P003 pronunciation-scoring pipeline

Not intended as:

• a general-purpose English ASR model
• a production transcription endpoint
• a standalone pronunciation assessor without the downstream GOP/GOPT pipeline

Important limitation:

• the trainer-side eval/per metric reported here is only the phoneme-CTC training metric used during fine-tuning
• the meaningful P003 research result is phone-level PCC on SpeechOcean762 after GOP/GOPT evaluation, which is tracked separately

Training and evaluation data

Training data:

• LibriSpeech 960h (train_clean_100, train_clean_360, train_other_500)

Trainer-side evaluation data:

• LibriSpeech validation split used by the Hugging Face Trainer

Research evaluation data:

• SpeechOcean762 is used later in the P003 eval sweep, not in this training run

Training procedure

This run used the shared P003 phoneme-head recipe and completed all planned training steps locally on March 6, 2026.

Selection policy:

• training ran to global_step = 13182
• the exported root artifact was selected from the best checkpoint
• best checkpoint by tracked trainer metric: checkpoint-8500
• best tracked trainer metric:
  • eval_per = 0.9988901220865705 at step 8500
• final trainer eval at step 13000:
  • eval_loss = 0.10874085873365402
  • eval_per = 0.9992600813910469

Artifact integrity:

• exported root model.safetensors matches local checkpoint-8500
• pushed Hugging Face model.safetensors matches that same artifact

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1318
• num_epochs: 3

Training results

These are trainer-side phoneme-CTC validation metrics, not final pronunciation scoring metrics.

Related runs

• W&B training run: https://wandb.ai/peacockery/hubert-base-phoneme-en/runs/qe7scuxw
• Canonical train sweep: projects/P003-compact-backbones/experiments/sweeps/final/train_hubert_base.yaml
• Canonical eval sweep: projects/P003-compact-backbones/experiments/sweeps/final/eval_hubert_base.yaml

Research status

• Training artifact: complete
• Hugging Face export: complete
• P003 pronunciation-scoring sweep: pending / separate from this card

Framework versions

• Transformers 5.2.0
• Pytorch 2.8.0+cu128
• Datasets 4.5.0
• Tokenizers 0.22.2