sarashina2.2-tts-oQ4

sarashina2.2-tts

sarashina2.2-tts is a Japanese-centric text-to-speech system built on a large language model, developed by SB Intuitions. It supports Japanese and English, delivering high pronunciation accuracy, naturalness, and stability across diverse speaking styles, with zero-shot voice generation support.

Highlights

• 🇯🇵 Japanese-Centric: Designed and optimized specifically for Japanese, with broad coverage of real-world use cases.
• 🎯 High Accuracy: Delivers strong pronunciation accuracy on Japanese text through large-scale end-to-end training.
• 🔒 Responsibly Sourced Training Data: Trained exclusively on legitimately acquired and properly licensed speech data.
• 🎙️ Zero-shot Voice Generation: Reproduces a speaker's voice, speaking style, and acoustic characteristics from a short reference clip.
• 🔊 Natural & Expressive: Produces highly natural speech with consistent quality, supporting a wide range of speaking styles including narration, broadcast, conversation, and customer service.
• 🌐 Bilingual: Supports both Japanese and English text-to-speech synthesis.

Training Data

This model was trained on audio data collected from legitimately purchased audio sources, public speech archives, and data gathered in compliance with applicable domestic laws. During collection, we adhered to robots.txt directives and terms of service to ensure proper data acquisition.

Usage

For installation instructions, Docker setup, and detailed usage, please refer to the GitHub repository.

Audio Samples

The samples below demonstrate key capabilities of sarashina2.2-tts:

• Speaking Style Variety: Transfers diverse speaking styles — narration, broadcast, conversation, customer service, and more — from reference audio.
• Zero-shot Voice Cloning: Reproduces a speaker's voice from just a few seconds of reference speech, with no fine-tuning required.
• Cross-lingual Generation: Preserves speaker identity and speaking style consistently across Japanese and English.
• Code Switching: Handles mixed Japanese-English sentences naturally within a single utterance.

Quick Sample

<table> <tr><td colspan="2"><b>Zero-shot Speaker Adaptation</b><br>東京から金沢までは新幹線を利用するのが便利で、所要時間は約２時間半です。</td></tr> <tr><th>Reference</th><th>Generated</th></tr> <tr> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/samples/prompt_A.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/samples/synthesized_A.wav"></audio></td> </tr> <tr><td colspan="2"><b>Diverse Speaking Styles</b><br>お待たせいたしました。お客様のSoftBank光のご契約状況が確認できました。あわせて、Y!mobileとのおうち割 光セットの適用状況をお調べしたいのですが、現在お使いの携帯電話番号をお伺いしてもよろしいでしょうか？</td></tr> <tr><th>Reference</th><th>Generated</th></tr> <tr> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/samples/prompt_C.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/samples/synthesized_C.wav"></audio></td> </tr> <tr><td colspan="2"><b>English Generation</b><br>There is something remarkable about the way language shapes the way we think. A single phrase, spoken in the right tone, can carry emotions that words alone cannot express.</td></tr> <tr><th>Reference</th><th>Generated</th></tr> <tr> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/samples/prompt_B.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/samples/synthesized_B.wav"></audio></td> </tr> </table>

Speaking Style Variety

<table> <tr><td colspan="2"><b>Narration</b><br>午前2時。東京・下町の一角。静まり返った住宅街に、リズミカルに包丁を叩く音が響く。店主の佐藤は、この場所で40年、変わらずにスープを炊き続けてきた。</td></tr> <tr><th>Reference</th><th>Generated</th></tr> <tr> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/styles/ナレーション_prompt.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/styles/ナレーション_synthesized.wav"></audio></td> </tr> <tr><td colspan="2"><b>Broadcast</b><br>国土交通省は15日、過疎地域や山間部における配送ルートの認可プロセスを簡略化する新指針を発表した。これにより、従来は数ヶ月を要していた飛行許可の申請期間が大幅に短縮される見通しだ。</td></tr> <tr><th>Reference</th><th>Generated</th></tr> <tr> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/styles/放送_prompt.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/styles/放送_synthesized.wav"></audio></td> </tr> <tr><td colspan="2"><b>Conversation</b><br>なるほど。じゃーちょっとすいませんそのー最近ハマってることについて、もう少しだけお話していただいてもいいですか？</td></tr> <tr><th>Reference</th><th>Generated</th></tr> <tr> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/styles/会話_prompt.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/styles/会話_synthesized.wav"></audio></td> </tr> <tr><td colspan="2"><b>Customer Service</b><br>お待たせいたしました。ご契約状況を確認したのですが、一点だけ補足で伺わせてください。ご登録いただいているお電話番号の下4桁、もしくはご生年月日を念のためお伺いしてもよろしいでしょうか？</td></tr> <tr><th>Reference</th><th>Generated</th></tr> <tr> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/styles/接客_prompt.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/styles/接客_synthesized.wav"></audio></td> </tr> <tr><td colspan="2"><b>Rakugo</b><br>「こんな安月給でやってられっかい、仕事なんかもう辞めたらあ！」て酒場で管巻いとったおっさんがな。翌朝になったら、誰よりもはよ店出て、鼻歌まじりに丁稚使いよる。</td></tr> <tr><th>Reference</th><th>Generated</th></tr> <tr> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/styles/落語_prompt.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/styles/落語_synthesized.wav"></audio></td> </tr> </table>

Zero-shot Voice Generation

<table> <tr><td colspan="3">東京から金沢までは新幹線を利用するのが便利で、所要時間は約２時間半です。</td></tr> <tr><th>Speaker</th><th>Reference</th><th>Generated</th></tr> <tr> <td>Speaker A (Male)</td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/zero_shot/prompt_A.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/zero_shot/synthesized_A.wav"></audio></td> </tr> <tr> <td>Speaker B (Female)</td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/zero_shot/prompt_B.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/zero_shot/synthesized_B.wav"></audio></td> </tr> <tr> <td>Speaker C (Female)</td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/zero_shot/prompt_C.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/zero_shot/synthesized_C.wav"></audio></td> </tr> <tr> <td>Speaker D (Senior Female)</td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/zero_shot/prompt_D.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/zero_shot/synthesized_D.wav"></audio></td> </tr> <tr><td colspan="3">The bullet train from Tokyo to Kanazawa takes approximately two and a half hours, making it the most convenient option for travel.</td></tr> <tr><th>Speaker</th><th>Reference</th><th>Generated</th></tr> <tr> <td>Speaker E (Female)</td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/zero_shot/prompt_en.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/zero_shot/synthesized_en.wav"></audio></td> </tr> <tr> <td>Speaker F (Male)</td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/zero_shot/prompt_en_m.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/zero_shot/synthesized_en_m.wav"></audio></td> </tr> </table>

Cross-lingual Zero-shot

<table> <tr><td colspan="2"><b>English Speaker to Japanese Generation</b><br>東京から金沢までは新幹線を利用するのが便利で、所要時間は約２時間半です。</td></tr> <tr><th>Reference</th><th>Generated</th></tr> <tr> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/cross_lingual/prompt_en.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/cross_lingual/en2ja.wav"></audio></td> </tr> <tr><td colspan="2"><b>Japanese Speaker to English Generation</b><br>The bullet train from Tokyo to Kanazawa takes approximately two and a half hours, making it the most convenient option for travel.</td></tr> <tr><th>Reference</th><th>Generated</th></tr> <tr> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/cross_lingual/prompt_ja.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/cross_lingual/ja2en.wav"></audio></td> </tr> </table>

Code Switching

<table> <tr><td colspan="2"><b>Mixed Japanese-English Sentence</b><br>最新のAI technologies、特にlarge language modelsは、音声合成の分野に大きなRevolutionをもたらしています。</td></tr> <tr><th>Reference</th><th>Generated</th></tr> <tr> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/code_switching/prompt_ja_A.wav"></audio></td> <td><audio controls controlsList="nodownload" src="https://huggingface.co/sbintuitions/sarashina2.2-tts/resolve/main/samples/code_switching/synthesized_codeswitching_A.wav"></audio></td> </tr> </table>

Acknowledgments

This model is built upon or incorporates code and models from the following open-source projects:

• CosyVoice
• HiFT-GAN
• 3D-Speaker

License

This model is licensed under Sarashina Model NonCommercial License Agreement.

If you are interested in using this model for commercial purposes, please feel free to contact us through our contact page.

The audio provided on this page are for research purposes only and may not be redistributed or used for commercial purposes.