speecht5_finetuned_voxpopuli_hy

Armenian SpeechT5 — HyVoxPopuli (speecht5_finetuned_voxpopuli_hy)

Primary production checkpoint for Eastern Armenian text-to-speech. Fine-tuned SpeechT5 on HyVoxPopuli and related Armenian speech data.

Model lineage

microsoft/speecht5_tts
  → Edmon02/speecht5_finetuned_hy (Common Voice 11)
  → Edmon02/speecht5_finetuned_voxpopuli_nl (intermediate)
  → Edmon02/speecht5_finetuned_voxpopuli_hy  ← you are here

For continued training with a larger vocabulary, use Edmon02/TTS_NB_2.

Quick start

import torch
import soundfile as sf
from datasets import load_dataset
from transformers import SpeechT5Processor, SpeechT5ForTextToSpeech, SpeechT5HifiGan

device = "cuda" if torch.cuda.is_available() else "cpu"
model_id = "Edmon02/speecht5_finetuned_voxpopuli_hy"

processor = SpeechT5Processor.from_pretrained(model_id)
model = SpeechT5ForTextToSpeech.from_pretrained(model_id).to(device)
vocoder = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan").to(device)

# Example speaker embedding from dataset (pick one speaker_id)
ds = load_dataset("Edmon02/hyvoxpopuli", split="train")
sample = ds[0]
speaker_embedding = torch.tensor(sample["speaker_embeddings"]).unsqueeze(0).to(device)  # if stored
# Or compute with SpeechBrain ECAPA — see project notebooks

inputs = processor(text="բարև, սա հայերեն խոսքի սինթեզի փորձ է։", return_tensors="pt")
inputs = {k: v.to(device) for k, v in inputs.items()}

with torch.no_grad():
    spectrogram = model.generate_speech(inputs["input_ids"], speaker_embedding, vocoder=vocoder)

sf.write("out.wav", spectrogram.cpu().numpy(), samplerate=16000)

Note: You must supply a speaker embedding (512-d) matching training. See projects/armenian-tts/notebooks/04_speecht5_hy.ipynb for the full inference pipeline.

Intended uses

• Armenian TTS inference and demos
• Fine-tuning starting point (prefer TTS_NB_2 for new tokenizer/vocab work)
• Research on low-resource TTS

Out of scope

• Real-time voice cloning without proper speaker embeddings
• Languages other than Armenian without retraining
• Production SLA without latency/quality benchmarking

Training data

• Edmon02/hyvoxpopuli — literary Armenian narration (~6 h, 623 segments)
• Common Voice hy-AM (notebooks) for augmentation
• Filter empty normalized_text rows before training (82 rows unlabeled in v1)

Training (summary)

Full logs were in local runs/ (removed from Hub for cleanliness).

Limitations

• Small domain: audiobook / literary style, 2 speakers
• 13% of HyVoxPopuli clips have no text — filter at train time
• Requires external vocoder (microsoft/speecht5_hifigan)
• Misleading repo name (voxpopuli_*) — data is not EU parliament VoxPopuli audio

Related assets

Citation

@misc{armenian_speecht5_hy2024,
  author = {Avetisyan, Edmon},
  title = {Armenian SpeechT5 (HyVoxPopuli fine-tune)},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Edmon02/speecht5_finetuned_voxpopuli_hy}}
}

License

MIT (model card). Dataset: CC-BY-4.0. Microsoft SpeechT5 weights subject to original license.