chhotu-turn-detection

Chhotu (छोटू): Tiny Hinglish Audio Turn Detection, From Scratch

▶ Try the live demo in your browser (the model runs client-side, your mic audio never leaves your device)

Chhotu answers the question every voice agent keeps asking: has the user finished speaking, or are they just pausing to think? It is built for Indian Hinglish conversations. Filler words like "achha", "toh", "matlab", "haan" and "umm" hold the turn open. A completed thought commits it, and the agent can respond.

Results

Both models scored on identical audio: the frozen smart-turn-data-v3.2 test set (10,878 clips), with Smart Turn v3.2 run through its own official inference path.

The honest reading: the pretrained teacher is ahead on every slice of the main test set, by about 6 points overall. That is what 4.5 million hours of speech pretraining buys, and we do not dress it up. Chhotu's case is the ratio: it lands within 3.7 points of the teacher on Hindi while using 40% of the parameters, half the file size and roughly 3.5x less compute, starting from random weights.

On a held-out Hinglish stress set (code-mixed sentences rendered by a different TTS engine), the picture is mixed and worth stating precisely:

Chhotu edges ahead on the held-out native Hindi voice, which is the case it was built for. On the deliberate accent-shift probe (English-accented voices reading Hindi script) it degrades badly while the 23-language teacher holds up. Both results are real and both are reported.

Why it's interesting

• Actually from scratch. The weights start from random initialization. The reference model in this space, Smart Turn v3.2, fine-tunes a pretrained Whisper-Tiny encoder. Chhotu starts from zero and still gets close.
• Tiny and fast. About 3.3M parameters, a 4.5MB INT8 ONNX file, and CPU latency around 9ms on a laptop. On identical hardware it runs about 3.3x faster than Smart Turn v3.2.
• One file to deploy. The ONNX graph takes a raw 16kHz waveform and returns p(turn complete). The log-mel frontend lives inside the graph, so inference needs nothing beyond onnxruntime and numpy.
• Distilled carefully. Trained with knowledge distillation from Smart Turn v3.2, using a confidence gate that drops the teacher's own mistakes so they don't get copied.
• Stress-tested on Hinglish. Evaluated on code-mixed clips generated with a different TTS engine and voices the model never heard in training.

Usage

import numpy as np, onnxruntime as ort

sess = ort.InferenceSession("chhotu_int8.onnx", providers=["CPUExecutionProvider"])

def p_turn_complete(wav_16k: np.ndarray) -> float:   # float32 mono @ 16kHz
    x = wav_16k[-128000:]                            # last 8 seconds
    x = (x - x.mean()) / (x.std() + 1e-7)            # standardize
    buf = np.zeros((1, 128000), np.float32)
    buf[0, 128000 - len(x):] = x                     # left-pad
    return float(sess.run(None, {"waveform": buf})[0])

In a real voice pipeline, run a cheap VAD such as Silero continuously. When it detects a pause of roughly 200ms, call Chhotu on the last 8 seconds. If the turn is complete, respond right away. If not, keep listening, with a hard timeout around 3 seconds as a safety net.

Training data and honest limitations

Chhotu was trained on the English, Hindi, Marathi and Bengali slices of pipecat-ai/smart-turn-data-v3.2 (about 81k clips), plus a self-generated code-mixed Hinglish TTS set. Worth knowing before you rely on the numbers:

• All Hindi, Marathi and Bengali audio in both train and test is synthetic TTS. That is a property of the best open corpus available today, and the reference model shares it. Real-world Indic speech performance is therefore unmeasured. The cross-TTS stress set is a proxy, not a substitute.
• Heavy accent shift is still hard. Indian-English voices reading Devanagari degrade both Chhotu and the reference model.
• Single-speaker audio only. Overlapping speech and backchannels are out of scope.

License and provenance

These weights were trained on the openly published pipecat smart-turn-data-v3.2 corpus (which carries no explicit license on its dataset card) and distilled from Smart Turn v3.2 (whose code is BSD-2-Clause). Out of respect for that chain, this repo does not claim a formal license of its own. Treat the weights as available for research, evaluation and learning; for commercial use, check the upstream terms yourself.

The whole project, from raw data to these weights, runs in a single reproducible Kaggle notebook: data prep, teacher pass, TTS synthesis, training, evaluation against Smart Turn v3.2 on identical clips, quantization and CPU benchmarks.

📓 The full Kaggle notebook (fork it and hit Run-All to reproduce everything from scratch)