LocalVQE

LocalVQE

Local Voice Quality Enhancement — compact neural models for acoustic echo cancellation (AEC), noise suppression (NS), and dereverberation of 16 kHz speech, running on commodity CPUs in real time. Causal and streaming (256-sample hop, 16 ms latency).

• Try it: https://huggingface.co/spaces/LocalAI-io/LocalVQE-demo
• Source, build system, tests: https://github.com/localai-org/LocalVQE

This page hosts the published weights. Inference runs the GGML C++ engine on the GGUF files directly (build instructions on GitHub).

Authors: Richard Palethorpe (richiejp) and Claude (Anthropic). LocalVQE is a streaming, CPU-tuned derivative of DeepVQE (Indenbom et al., Interspeech 2023).

Models

Speed is per 16 ms hop on a Ryzen 9 7900 (Zen4), 4 threads; RT = realtime factor (higher is faster than realtime).

• Joint models (v1.2 / v1.3) clean echo, noise, and reverb in one pass. v1.3 is wider and filters noise better; v1.2 is ~1/4 the per-hop cost.
• v1.4-AEC removes only the far-end echo and passes voice, room, and background through unchanged. It's a classical adaptive filter followed by a small neural mask. The 2.7K build is that filter alone — cheaper and gentler, but it can't remove heavily reverberant echo the way the mask can.
• Every model needs a far-end reference signal (a loopback of what your speakers play) in addition to the mic.
• bf16 GGUFs are ~12 % smaller with identical quality and speed; pick f32 unless download size matters.

Compact line — GTCRN-AEC (for lower-power CPUs)

A separate, much smaller second line of models for lower-power CPUs: a ~49 K-parameter GTCRN-AEC network — a distinct architecture based on GTCRN (Rong et al., ICASSP 2024) — paired with the project's DSP echo-cancellation front-end. The GGUFs are self-contained, so they run with the same single command as every other model. Two variants share the architecture:

Whole-clip real-time factor on the real ggml graph, benchmarked on a Raspberry Pi 5 (one example of a low-power target; test_gtcrn --bench, Cortex-A76, Ubuntu 24.04), parity-verified to the PyTorch reference within ~1e-6 on-device. RTF is identical for both variants:

That is ~0.78 ms per 16 ms hop single-threaded. Runs on any CPU; for single-board ARM, cross-compile for aarch64 with ggml/docker/Dockerfile.arm64 (docker buildx

• qemu). f16/q8 quantizations are published only if/when released.

Files in this repository

v1.4-AEC is GGUF-only (no .pt). GGUF integrity is checked at load time against a built-in SHA256 allowlist in the engine.

Performance

Full 800-clip eval on the ICASSP 2022 AEC Challenge blind test set (real recordings). AECMOS echo / deg are 1–5 (higher = more echo removed / cleaner speech); blind ERLE is 10·log10(E[mic²]/E[enh²]), only meaningful on far-end-only clips. Unprocessed-mic echo MOS is 2.67 / 2.56 / 1.90 / 2.13 / 5.00 across the five scenarios.

v1.4-AEC — keeps background noise and room by design, so its ERLE and far-end DNSMOS are intentionally lower than the joint models (it isn't deleting the ambience):

v1.4-AEC 2.7K (front-end only) — matches or beats the full model's perceptual far-end echo at 1/74 the parameters; the mask's extra work shows up as higher ERLE above, not higher echo MOS:

v1.3 (joint) and v1.2 (joint) — these also delete the background, so their far-end ERLE is much higher and not comparable to v1.4-AEC's:

Latency

Per-hop p50 / RT factor on a Ryzen 9 7900 (Zen4). 16 kHz, 256-sample hop.

¹ v1.4-AEC's adaptive front-end always runs on CPU and the neural stage is too small for GPU offload to pay off. Four threads is the sweet spot on Zen4 for all models; the library defaults to min(4, available CPUs).

Memory (CPU)

Working set the model adds on top of the ~7 MiB binary baseline:

Running inference

Download a GGUF (web UI, huggingface-cli, or hf_hub_download) and run the GGML CLI — same command for every model, just swap the file:

./localvqe localvqe-v1.3-4.8M-f32.gguf --in-wav mic.wav ref.wav --out-wav out.wav

16 kHz mono PCM for both the mic and the far-end reference. Building the engine, the C API (liblocalvqe.so), and the OBS Studio plugin are documented in the GitHub repository.

PyTorch reference

localvqe-v1.3-4.8M.pt and localvqe-v1.2-1.3M.pt are the checkpoints used to produce the GGUF exports — for verification, ablation, and research, not end-user inference (use the GGML build). The model definition lives under pytorch/ in the GitHub repo.

Citing

Cite the repository via CITATION.cff at https://github.com/localai-org/LocalVQE (GitHub's "Cite this repository" button produces APA / BibTeX), and the upstream DeepVQE paper:

@inproceedings{indenbom2023deepvqe,
  title     = {DeepVQE: Real Time Deep Voice Quality Enhancement for Joint
               Acoustic Echo Cancellation, Noise Suppression and Dereverberation},
  author    = {Indenbom, Evgenii and Beltr{\'a}n, Nicolae-C{\u{a}}t{\u{a}}lin
               and Chernov, Mykola and Aichner, Robert},
  booktitle = {Interspeech}, year = {2023},
  doi       = {10.21437/Interspeech.2023-2176}
}

The compact GTCRN-AEC line is based on GTCRN — please also cite:

@inproceedings{rong2024gtcrn,
  title     = {GTCRN: A Speech Enhancement Model Requiring Ultralow
               Computational Resources},
  author    = {Rong, Xiaobin and Sun, Tianchi and Zhang, Xu and Hu, Yuxiang
               and Zhu, Changbao and Lu, Jing},
  booktitle = {ICASSP 2024 - 2024 IEEE International Conference on Acoustics,
               Speech and Signal Processing (ICASSP)},
  pages     = {971--975}, year = {2024},
  doi       = {10.1109/ICASSP48485.2024.10448310}
}

Dataset attribution

Weights are trained on the ICASSP 2023 DNS Challenge (Microsoft, CC BY 4.0) and fine-tuned on the ICASSP 2022/2023 AEC Challenge.

Safety

Training data was filtered by DNSMOS, which can misclassify distressed speech (screaming, crying) as noise. LocalVQE may attenuate such signals and must not be relied upon for emergency or safety-critical applications.

License

Apache License 2.0.