hifigan-wavlm-mlx

HiFi-GAN (WavLM-conditioned, bshall/knn-vc) — for MLX-Swift

Pre-converted weights of the WavLM-conditioned HiFi-GAN generator from bshall/knn-vc (the prematched variant) for the HiFiGANWavLMMLX Swift package.

This is not a generic mel-spectrogram HiFi-GAN. Input is WavLM-Large hidden states (B, T, 1024) — typically layer 6 — and output is 16 kHz audio at T × 320 samples. Companion to gwenn-ha-dev/wavlm-large-mlx.

• Format: safetensors, float32
• Size: 63 MB, 158 entries, 16.52 M parameters
• SHA-256: 4e3b96fa86e9b18b0062959b72e83b2e247d43f27a8a5e673f8adf72f1b5ca09
• Variant: prematched (k=4 prematching), 16 kHz output

What's in this repo

Difference vs upstream bshall/knn-vc

Upstream ships the generator as a PyTorch .pt checkpoint (prematch_g_02500000.pt). Every Conv1d and ConvTransposed1d uses weight_norm, so each conv stores two tensors (weight_g, weight_v) instead of a single weight.

This .safetensors ships with all weight_norm materialized:

weight = weight_g · weight_v / ‖weight_v‖

Mathematically equivalent at inference (no numerical change), but lets MLX-Swift use plain Conv1d / ConvTransposed1d. State-dict entry count drops from 234 (raw) to 158 (materialized).

The materialization is a one-line PyTorch op on the loaded generator:

generator.remove_weight_norm()

Numerical parity vs PyTorch reference

Measured against the PyTorch reference on three deterministic fixtures, in float32, on M3 Max:

L1 is what matters for perceived audio quality. The looser L_∞ peaks come from float32 accumulation across ~30 dilated Conv1d + 4 ConvTransposed1d; they correspond to isolated samples and are audible nowhere. See the Swift package's port notes for the full methodology.

Usage

See the HiFiGANWavLMMLX Swift package for full integration. Three lines:

let arrays = try SafetensorsLoader.load(url: url)
let hifigan = HiFiGAN()
try hifigan.loadWeights(from: arrays)

For a full kNN-VC pipeline, pair with gwenn-ha-dev/wavlm-large-mlx:

let hidden = wavlm(waveform)[6]            // (1, T, 1024)
let audio  = hifigan(hidden)               // (1, T*320)

To download from this repo:

huggingface-cli download gwenn-ha-dev/hifigan-wavlm-mlx \
    hifigan-wavlm.safetensors --local-dir ./weights

License

Code (the Swift package): MIT.

Weights derive from bshall/knn-vc (MIT), itself derived from the upstream HiFi-GAN reference by Kong et al. (MIT).

Citations

@inproceedings{Kong2020HiFiGAN,
  title={HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis},
  author={Kong, Jungil and Kim, Jaehyeon and Bae, Jaekyoung},
  booktitle={Advances in Neural Information Processing Systems},
  year={2020}
}

@inproceedings{Baas2023kNNVC,
  title={Voice Conversion With Just Nearest Neighbors},
  author={Baas, Matthew and van Niekerk, Benjamin and Kamper, Herman},
  booktitle={Interspeech},
  year={2023}
}