BiMTokenizer

BiMTokenizer

BiMTokenizer is a bidirectional Mamba speech tokenizer for low-bitrate neural speech coding. This repository provides five 16 kHz checkpoints from the same model family.

• Paper: BiMTokenizer: Preserving Semantic-Acoustic Balance in Low-Bitrate Speech Tokenization via Bidirectional State-Space Modeling
• Code: ZhangXinWhut/BiMTokenizer
• Audio demo: BiMTokenizer Demo

Authors

Xin Zhang, Lin Li, Chuanbo Liu, Jianquan Liu, and Kong Aik Lee.

Available checkpoints

Each checkpoint is stored next to its matching config.yaml. Do not mix a checkpoint with a configuration from another variant.

LibriSpeech test-clean results

Download

Install the Hub client:

pip install -U huggingface_hub

Download one model variant together with its configuration and the repository manifest. Including the root config.yaml also allows Hugging Face to count the download at repository level.

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="ZhangXinWhut/BiMTokenizer",
    allow_patterns=["config.yaml", "whisper/*"],
    local_dir="./BiMTokenizer-Whisper",
)

Replace whisper/* with sensevoice/*, sensevoice-32768-4096/*, sensevoice-2048/*, or sensevoice-2048-emilia2w/* for the other checkpoints.

The complete repository can also be downloaded with:

huggingface-cli download ZhangXinWhut/BiMTokenizer \
  --local-dir ./BiMTokenizer-weights

Inference

Clone the code repository and place the downloaded checkpoint under ./weights/. Example:

git clone https://github.com/ZhangXinWhut/BiMTokenizer.git
cd BiMTokenizer

python inference.py \
  --config_path /path/to/whisper/config.yaml \
  --checkpoint_path /path/to/whisper/bimtokenizer_whisper_librispeech.pt \
  --input_dir /path/to/input_wavs \
  --output_dir output_wavs \
  --device cuda \
  --batch_size 1

The fixed Leech codebooks required by the configurations are included in the GitHub code repository under bimtokenizer/modules/quantizer/cache/.

Notes

• All released checkpoints operate at a 16 kHz sample rate.
• The public download counter is repository-level and does not identify individual downloaders.
• Download statistics may take time to appear after the first downloads.