DhVaani-0.5-MNN

DhVaani-0.5 — MNN (Android / on-device)

DhVaani-0.5 converted to MNN for fully offline, on-device text-to-speech. Ships with a complete, working Android integration — C++ engine, JNI bridge, Kotlin API and a runnable sample app in android/.

v1.1 — models were rebuilt. The .mnn files published before 2026-09-01 produced silence. If you pulled this repo earlier, re-download. Details in What was wrong.

---

Quick start (Android)

git clone https://huggingface.co/Bbkblo/DhVaani-0.5-MNN
cd DhVaani-0.5-MNN/android

./tools/fetch_mnn.sh        # official MNN 3.6.1 prebuilt .so + headers
./tools/fetch_models.sh     # models into sample assets
cp ~/my_voice.wav sample/src/main/assets/dhvaani/prompt.wav
./gradlew :sample:installDebug

val tts = DhVaani.fromAssets(context)
tts.setPrompt(promptPcm, 24000, "exactly what the reference clip says")
val audio = tts.synthesize("नमस्ते दुनिया")   // FloatArray, 24 kHz mono
tts.play(audio!!)

Full guide: ANDROID_SETUP.md

---

Files

Runtime footprint: ~140 MB of assets.

---

Graph interfaces

Two things that bite people:

1. All graphs contain subgraphs — they only run through MNN::Express::Module. The stock MNNNetInstance Java API exposes only the legacy Interpreter/Session path and cannot run these models. That's why this repo ships its own JNI layer.
2. Scalars must be rank-0, shape {} and not {1}.

The mel filterbank, the vocos output projection and the ISTFT are not in the graphs — they are implemented in android/dhvaani/src/main/cpp/dhvaani.cpp.

---

Verification

Measured, not assumed. Reproduce with dhvaani_mnn.py and android/tools/cli_test.cpp.

MNN vs fp32 ONNX, identical inputs:

C++ engine components vs the Python reference:

Android build (NDK r27c, arm64-v8a, android-24): libdhvaani.so 1.1 MB, 9/9 JNI symbols exported, 8/8 undefined MNN symbols resolve against the official prebuilt libMNN_Express.so. armeabi-v7a builds too.

Sample output: test/dhvaani_mnn_cpp.wav.

---

What was wrong in v1.0

The original .mnn files were converted from the INT8 ONNX exports. MNNConvert does not correctly fold ONNX QDQ (QuantizeLinear/DequantizeLinear) nodes in these graphs. The models loaded and ran without any error, but the numbers were garbage:

The flow decoder's output barely changed with t, mel values came out in the range [-132, +187] instead of [-7, +4], and the rendered audio was silence (RMS 0.0001, zero voiced frames).

Fix: convert from the fp32 ONNX and let MNN do its own weight quantization. Same file size, numerically faithful:

mnnconvert -f ONNX --modelFile fm_decoder.onnx \
    --MNNModel fm_decoder_int8.mnn --bizCode fm \
    --weightQuantBits 8 --weightQuantAsymmetric

See android/tools/convert.sh.

---

Python usage

pip install mnn==3.6.1 numpy soundfile scipy

python3 dhvaani_mnn.py \
    --text "आपका टेक्स्ट" \
    --prompt-wav prompt.wav --prompt-text "prompt ka text" \
    --dir . --out out.wav --num-step 8 --threads 4

---

Notes on quality

• Reference clip: 3–10 s of clean speech, and promptText must match it word for word. This matters more than any inference setting.
• numStep is the quality/speed dial and dominates runtime (cost is linear). 4 = rough, 8 = balanced, 16 = best.
• Synthesize sentence by sentence and start playback on the first one — it cuts perceived latency far more than any parameter tuning.

Source

• ONNX: https://huggingface.co/Bbkblo/DhVaani-0.5-ONNX
• MNN: https://github.com/alibaba/MNN