LocalVQE-v1.4-AEC-200K-CoreML

LocalVQE v1.4-AEC 200K Core ML

Stateful FP16 Core ML export of the learned residual echo mask from LocalVQE v1.4-AEC. It is designed for local, streaming acoustic echo cancellation on Apple hardware.

This is an echo-only hybrid. It is not the older LocalVQE v1.3 joint noise-and-echo enhancer, and the Core ML model is not a standalone waveform model. A host implementation must run delay estimation, adaptive echo filtering, the short controller network, FFT, and overlap-add around it.

Model

Files

No source GGUF is duplicated here. The verified upstream source remains in the LocalVQE repository.

Performance

Measured on an Apple M5 Pro. Each frame represents 16 ms of audio. Lower latency and real-time factor are better; higher audio-to-wall-time is better.

The complete-hybrid number conservatively adds separately measured host and Core ML stages. It is not yet an integrated Swift measurement.

The Core ML mask had 1.356% relative spectral error against the reconstructed PyTorch reference on the public ten-second double-talk example. Its output was 37.68 dB SI-SDR relative to the PyTorch waveform.

Echo quality

Microsoft AECMOS v4 scores the same public double-talk recording on a one-to-five scale. Higher is better for both columns.

On this recording, the adaptive filter alone removes slightly more echo but damages near-end speech more. The learned mask restores 0.194 MOS of speech quality while giving back 0.077 echo MOS. Core ML changes the two PyTorch scores by only 0.0002 and 0.0022.

Download

hf download aufklarer/LocalVQE-v1.4-AEC-200K-CoreML \
  --local-dir LocalVQE-v1.4-AEC-200K-CoreML

Python usage

The two inputs are spectra produced by the host adaptive-filter front end: the residual microphone signal e and estimated echo yhat. Both have shape (1, 2, 1, 256), containing real and imaginary values for FFT bins 1 through 256. Reset the Core ML state at the beginning of every audio session.

from pathlib import Path

import coremltools as ct
import numpy as np
from huggingface_hub import snapshot_download

root = Path(snapshot_download(
    "aufklarer/LocalVQE-v1.4-AEC-200K-CoreML"
))
model = ct.models.CompiledMLModel(
    str(root / "LocalVQEAECResidualMask.mlmodelc"),
    compute_units=ct.ComputeUnit.ALL,
)
state = model.make_state()

# Supply one spectrum from each host-DSP stream every 16 ms.
residual_spectrum = np.zeros((1, 2, 1, 256), dtype=np.float16)
echo_estimate_spectrum = np.zeros((1, 2, 1, 256), dtype=np.float16)
enhanced_spectrum = model.predict(
    {
        "mic_spectrum": residual_spectrum,
        "reference_spectrum": echo_estimate_spectrum,
    },
    state=state,
)["enhanced_spectrum"]

The adaptive front end follows upstream LocalVQE v1.4: GCC-PHAT delay acquisition, a partitioned-block frequency-domain Kalman filter, and its v2xp controller. LocalVQEAECFrontend.npz contains the controller tensors and exact DSP constants required by a native implementation.

Limitations

• This export removes playback echo. It is not a general noise suppressor.
• Quality was measured on one public double-talk example. Evaluate a larger AEC corpus and recordings from the intended capture hardware before deployment.
• The host adaptive filter is required; passing microphone and playback spectra directly to the Core ML mask does not implement the published system.
• The complete native Swift pipeline and its memory use have not yet been measured.

Source and license

Converted from the F32 localvqe-v1.4-aec-200K release, run v1.4.r005, with source SHA-256 b6e43138588a83bfe903ab5e143b4020b91c1e1629f5a575ac5855ff0003c731.

• LocalVQE source and documentation
• Upstream model files
• Microsoft AECMOS

The upstream implementation and weights are Apache-2.0 licensed.

Links

• speech-swift — Apple SDK
• Documentation — installation and CLI
• soniqo.audio — website
• Blog — updates and technical notes