DEntist-spark

Model Card for spark-tts-egyptian-dental-clinic-voice

A LoRA adapter that fine-tunes an Arabic Spark-TTS checkpoint to speak Egyptian Arabic in the voice of a single dental-clinic speaker, trained on real clinic recordings.

Model Details

Model Description

This model is a LoRA adapter on top of MrEzzat/Spark_TTS_Arabic (itself a fine-tune of SparkAudio/Spark-TTS-0.5B), further adapted on ~600 real recordings from an Egyptian dental clinic so the base model's Arabic TTS ability speaks in that clinic's own voice and dialect rather than generic Modern Standard Arabic.

• Developed by: Noureldin Ayman
• Model type: Text-to-speech — LoRA adapter for an LLM-based TTS model (causal LM + BiCodec neural audio codec)
• Language(s): Egyptian Arabic (dental-clinic domain)
• License: cc-by-nc-sa-4.0 — inherited from the base model; non-commercial use only, derivatives must be released under the same license
• Finetuned from model: MrEzzat/Spark_TTS_Arabic

Model Sources

• Repository: github.com/nourayman21k/DEntist-Spark
• Base model: MrEzzat/Spark_TTS_Arabic
• Base architecture paper: Spark-TTS (arXiv:2503.01710)

Uses

Direct Use

Given a text prompt (Egyptian Arabic, dental-clinic phrases — greetings, appointment booking/rescheduling, confirmations, procedures, symptoms, aftercare instructions, pricing) and a short reference clip for speaker tokens, the model generates speech in the trained clinic voice. It is intended for that clinic's own voice assistant / IVR / appointment-related speech, not as a general-purpose Arabic TTS model.

Downstream Use

Could be embedded in an appointment-booking voice bot or phone-line IVR system for the clinic whose recordings were used for training.

Out-of-Scope Use

• Cloning this voice for anything other than the original clinic use case, without the speaker's consent.
• General-purpose or open-domain Arabic TTS — the model was trained on a narrow domain and has not been evaluated outside it.
• Commercial use — the base model's CC BY-NC-SA 4.0 license is non-commercial only.
• Impersonation, fraud, deepfakes, or any deceptive use, per the base model's usage restrictions.

Bias, Risks, and Limitations

• Single-speaker voice clone. This adapter reproduces one specific person's voice. Treat generated audio with the same consent and disclosure considerations as any voice clone.
• Narrow training domain. The training text is almost entirely undiacritized (diacritics cover roughly 0.2% of characters), while the base checkpoint was trained on fully diacritized Modern Standard Arabic. Pronunciation is noticeably weaker on sentences well outside the dental-clinic training distribution.
• Utterance-initial words are occasionally unclear, likely related to the silence-trimming step used during data preparation.
• Generation-time looping. Greedy decoding can occasionally get stuck in a repeating token cycle instead of emitting a stop token near the generation cap; mitigated with repetition_penalty / no_repeat_ngram_size at inference time.
• No formal automatic evaluation (WER, MOS, speaker-similarity) has been run yet — assessment so far is qualitative (listening tests and a memorization check).

Recommendations

Listen-check output before relying on it, especially for text far from the training domain, and do not use this adapter to synthesize this voice for purposes beyond the original clinic use case without the speaker's consent.

How to Get Started with the Model

This is a PEFT/LoRA adapter, not a standalone model — you need the base checkpoint, the adapter, and the BiCodec audio tokenizer from the Spark-TTS repo.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE_MODEL = "MrEzzat/Spark_TTS_Arabic"  # or a local snapshot_download of it
ADAPTER    = "nour21k/spark-tts-egyptian-dental-clinic-voice-v3"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
base_model = AutoModelForCausalLM.from_pretrained(BASE_MODEL, torch_dtype=torch.float32)
model = PeftModel.from_pretrained(base_model, ADAPTER).to("cuda").eval()

You'll also need BiCodecTokenizer from the SparkAudio/Spark-TTS repo to turn text plus a reference audio clip into the token prompt, and to decode the generated semantic tokens back into a waveform. See dental-voice-load-and-test.ipynb in the project repository for the full reference-conditioned generation loop.

Training Details

Training Data

~600 real Egyptian-Arabic dental-clinic recordings (24kHz mono), scripted across categories such as greetings, patient registration, appointment booking/rescheduling, cancellations, confirmations, procedures & vocabulary, symptoms, doctor instructions & aftercare, pricing, filler words, and numbers/dates/times. Mixed in with a filtered subset (up to 1,200 clips, 1.5–12s duration, ≥8 characters of transcript) of the public ahmedshafiq12/egyptian-audio-dataset-collected-from-youtube Kaggle dataset for broader Egyptian-dialect coverage. The clinic's own recordings were oversampled 3× so they weren't outnumbered by the noisier YouTube-sourced dialect data. The recordings and transcripts themselves are private and are not distributed with this model.

Training Procedure

Preprocessing

Each recording was silence-trimmed (top_db=40), edge-padded (0.15s), peak-normalized, and resampled to the BiCodec sample rate, then tokenized into interleaved <bicodec_global_*> (speaker) / <bicodec_semantic_*> (content) token sequences via BiCodec. Before any tokenization, a round-trip encode→decode gate (minimum similarity score 0.35 against the original audio) had to pass, to catch a broken codec before it could corrupt the whole dataset.

Training Hyperparameters

• Training regime: fp32 (full precision — this checkpoint produces no audio tokens under fp16)
• Method: LoRA, r=16, alpha=16, dropout=0.0
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Epochs: 3
• Learning rate: 2e-4, linear schedule, 10 warmup steps
• Batch size: 2 per device × 4 gradient accumulation steps = effective batch size 8
• Weight decay: 0.001 · Optimizer: AdamW (torch)
• Gradient checkpointing: enabled
• Max sequence length: 2048 tokens

Speeds, Sizes, Times

Not logged for this run. Trained on a single Kaggle-provided GPU (P100/T4 class). [More Information Needed] for exact wall-clock training time.

Evaluation

Testing Data, Factors & Metrics

No held-out test split or automatic metric (WER, MOS, speaker similarity) has been computed. Evaluation so far has been qualitative:

• A codec round-trip gate confirms BiCodec can faithfully reproduce real speech before training starts.
• A pre-training sanity check decodes the exact training tokens back to audio to catch data-pipeline bugs early.
• A post-training memorization check (regenerating a sentence the model trained on) confirmed correct text→content mapping.
• Ad hoc listening tests on clinic phrases.

Results

Generated speech is genuine voiced output (correct pitch/prosody, syllable structure) rather than noise, and correctly renders words for sentences close to the training distribution. Known weak points: occasional unclear utterance-initial words, and reduced accuracy on sentences well outside the dental-clinic domain. No numeric scores are available.

Summary

Experimental / work in progress. Usable for qualitative demos within the trained domain; not yet validated for production deployment.

Environmental Impact

• Hardware Type: Single Kaggle-provided GPU (P100 or T4 class)
• Hours used: [More Information Needed]
• Cloud Provider: Kaggle
• Compute Region: [More Information Needed]
• Carbon Emitted: [More Information Needed]

Technical Specifications

Model Architecture and Objective

Spark-TTS architecture: a causal language model predicts discrete BiCodec audio tokens (global "speaker" tokens + semantic "content" tokens) conditioned on input text, and BiCodec decodes those tokens into a waveform. This repository holds only a LoRA adapter for the language-model component of MrEzzat/Spark_TTS_Arabic; the BiCodec codec itself is unmodified.

Compute Infrastructure

Hardware

Kaggle Notebooks, single GPU (P100/T4 class)

Software

Python 3.12 · transformers==4.56.2 · peft · accelerate · datasets · huggingface_hub<1.0 · librosa · soundfile · openpyxl · SparkAudio/Spark-TTS (for BiCodecTokenizer)

Citation

This work builds on the Spark-TTS architecture and the MrEzzat/Spark_TTS_Arabic checkpoint. Please cite the original Spark-TTS paper:

BibTeX:

@misc{wang2025sparktts,
  title={Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens},
  author={Xinsheng Wang and Mingqi Jiang and Ziyang Ma and Ziyu Zhang and Songxiang Liu and Linqin Li and Zheng Liang and Qixi Zheng and Rui Wang and Xiaoqin Feng and Weizhen Bian and Zhen Ye and Sitong Cheng and Ruibin Yuan and Zhixian Zhao and Xinfa Zhu and Jiahao Pan and Liumeng Xue and Pengcheng Zhu and Yunlin Chen and Zhifei Li and Xie Chen and Lei Xie and Yike Guo and Wei Xue},
  year={2025},
  eprint={2503.01710},
  archivePrefix={arXiv},
  primaryClass={cs.SD},
  url={https://arxiv.org/abs/2503.01710},
}

APA:

Wang, X., Jiang, M., Ma, Z., Zhang, Z., Liu, S., Li, L., Liang, Z., Zheng, Q., Wang, R., Feng, X., Bian, W., Ye, Z., Cheng, S., Yuan, R., Zhao, Z., Zhu, X., Pan, J., Xue, L., Zhu, P., Chen, Y., Li, Z., Chen, X., Xie, L., Guo, Y., & Xue, W. (2025). Spark-TTS: An efficient LLM-based text-to-speech model with single-stream decoupled speech tokens. arXiv:2503.01710.

Glossary

• BiCodec — the neural audio codec Spark-TTS uses to turn waveforms into discrete "global" (speaker) and "semantic" (content) tokens, and back into audio.
• LoRA (Low-Rank Adaptation) — a parameter-efficient fine-tuning method that trains small adapter matrices instead of updating the full model, used here to adapt the base model without erasing its existing Arabic ability.

More Information

See the project repository for the full data-preparation and training notebook, including the codec-gate and sanity-check steps: github.com/nourayman21k/DEntist-Spark

Model Card Authors

Noureldin Ayman

Model Card Contact

[More Information Needed]