subly-ru-tts

Subly RU TTS — On-device Russian voice "Natasha" (StyleTTS2 → Core ML)

A fully on-device Russian text-to-speech voice for Apple platforms (iOS/macOS), plus everything needed to turn raw Russian text into the phoneme stream the model expects. The acoustic model is a StyleTTS2 fine-tune exported to Core ML, designed to run through the FluidAudio StyleTTS2 runtime. No server, no network at inference time, no Python.

Built for the Subly app, released openly so anyone can use it. 24 kHz, female voice, neutral narration style.

---

What's in this repo

---

The pipeline

Russian text becomes audio in four steps. The first three are CPU-cheap; only the last touches the neural models.

raw text
  → ё-restoration         (ru_yo.bin)          "елка" → "ёлка"
  → stress marking        (ru_stress.bin,      "ёлка" → "+ёлка"   (+ before the stressed vowel)
                           OOV model for misses)
  → grapheme→IPA          (ru_g2p.py)          "+ёлка" → "jˈolkə"
  → StyleTTS2 synthesis   (models/, ref clip)  IPA → 24 kHz waveform

This mirrors the training pipeline (RUAccent stress + a rule-based G2P), so the IPA fed at inference matches what the model was trained on.

Recommended runtime parameters

Using it with FluidAudio (Swift)

import FluidAudio

// Point the manager at the `models/` directory from this repo.
let manager = StyleTTS2Manager(directory: modelsDir, computeUnits: .cpuOnly)
try await manager.initialize()

let ipa = "prʲɪvʲˈɛt ! ˈɛtə tʲˈɛst rˈusskəvə dʲˈiktərə ."   // from the pipeline above
let samples = try await manager.synthesize(
    ipa: ipa,
    referenceAudioURL: refClipURL,   // ru_natasha_ref.wav
    alpha: 0.5, beta: 0.9, noiseSeed: 1
)   // -> [Float] PCM @ 24 kHz

The IPA is tokenized by FluidAudio's StyleTTS2TextCleaner (the StyleTTS2 178-symbol set); any symbol outside that set is dropped, so feed clean IPA.

---

File formats (language-agnostic)

So you can read these from any language, not just Swift.

ru_stress.bin and ru_yo.bin

Little-endian, sorted by UTF-8 byte order (so a byte-wise binary search works):

ru_stress.bin:  ["RUS1"][u32 count][offsets: count × u32][records]
  record = [u8 keyLen][key UTF-8][u8 pos]
  pos = char index of the stressed vowel (where '+' is inserted); 255 = unstressed.

ru_yo.bin:      ["RUYO"][u32 count][offsets: count × u32][records]
  record = [u8 keyLen][key UTF-8][u8 posCount][posCount × u8 charIndex]
  charIndex = positions in the (lowercased) word where 'е' should become 'ё'.

offsets[i] is the byte offset of record i relative to the start of the records section (= 8 + count*4). Binary-search by reading offsets[mid], then the key.

ru_accent_oov_fp32.mlmodelc (OOV stress model)

Char-level token classifier (converted from RUAccent's nn_accent).

• Input input_ids: [1, 48] int32 — [bos]=2, then chars by the vocab below, [eos]=3, padded with [pad]=0.
• Output logits: [1, 48, 3] — labels 0=NO, 1=STRESS_PRIMARY, 2=STRESS_SECONDARY.
• Softmax per position; mark a + before char i-1 when position i is argmax PRIMARY with probability ≥ 0.55.

Vocab (id → token): 0 [pad], 1 [unk], 2 [bos], 3 [eos], 4 ', 5 -, 6 ., 7 ?, 8 `, 9 c, 10 e, 11 ́ (U+0301), then Cyrillic 12 а … 43 я, 44 ё.

---

Model details

• Architecture: StyleTTS2 (LibriTTS config, HiFi-GAN decoder), multilingual PL-BERT (ru), 80-mel / n_fft 2048 / win 1200 / hop 300, 24 kHz.
• Voice / training data: SOVA Dataset — TTS Natasha (~12.4 h, one female speaker).
• Phonemization: stress from RUAccent (dictionary + char model + ё-restoration), then a rule-based G2P covering palatalization, two-degree vowel reduction, iotation, final devoicing and regressive voicing assimilation, plus the г→в genitive exception.
• Core ML export: all stages FLOAT32 (the diffusion sampler overflows in FP16; other stages were kept FP32 for fidelity). Token-axis buckets 57/64/128/256.

---

Limitations

• Homographs (за́мок / замо́к, по́лет / полёт) take the dictionary's default stress — there is no context disambiguator here (that model is ~350 MB and was left out). They're rare in narration.
• Numbers/currency are not spelled out — normalize digits to words upstream (a bare 123 won't be voiced).
• Size: FP32 models are ~867 MB. For shipping, an FP16 re-export of the non-diffusion stages cuts this substantially, but re-validate quality by ear.
• CPU-only at inference (see above).

---

License & attribution

This repository is released under CC BY 4.0 — use it commercially or non-commercially, just keep the attributions below.

• Voice / reference audio derive from the SOVA Dataset (TTS Natasha), © Virtual Assistant, LLC, licensed CC BY 4.0.
• StyleTTS2 — MIT (Aaron (Yinghao) Li et al.).
• multilingual PL-BERT — MIT (Papercup).
• RUAccent (stress dictionary, OOV model, ё-dictionary are derived from it) — Apache-2.0 (Den4ikAI).

Please credit "SOVA TTS Natasha (CC BY 4.0, Virtual Assistant LLC)" and this repo when you use the voice.