FlexiSLM-0_5B-Stage2

FlexiSLM-0.5B-Stage2

Stage 2 checkpoint of FlexiSLM-0.5B, a compact spoken language model with dynamic and controllable frame rates on both speech input and output.

• Paper: arXiv:2606.31247
• Demo: flexislm.github.io
• Code: AmphionTeam/FlexiSLM
• Sibling checkpoint: FlexiSLM-7B-Stage2

This project is in active development. Checkpoints may be overwritten as training continues.

Quick start (auto-download)

Install from the code repo, then run with auto_download=True and checkpoint="stage2_0.5B". On first run this downloads this Stage 2 checkpoint plus the shared Qwen2.5-Omni audio encoder, SenseVoice, FlexiCodec, flow-matching decoder, and vocoder into models/.

git clone --recurse-submodules https://github.com/AmphionTeam/FlexiSLM.git
cd FlexiSLM
pip install -r requirements.txt

from pathlib import Path

import soundfile as sf
import torch

from src.inference_flexislm import FlexiSLMInferenceConfig, FlexiSLMInference

config = FlexiSLMInferenceConfig(
    auto_download=True,
    checkpoint="stage2_0.5B",  # this repo
    use_flow_matching_decoder=True,
    flow_matching_prompt_audio_path=str(Path("examples/input.wav").resolve()),
    enable_flexible_framerate=True,
    input_framerate=8.0,
    default_framerate=8.0,
    decode_audio=True,
    torch_dtype="bfloat16",
    attn_implementation="flash_attention_2",
)
engine = FlexiSLMInference(config, device="cuda:0")


def save_audio(result, output_path):
    waveform = result.get("audio")
    if waveform is None:
        raise RuntimeError("The model did not return decoded audio")
    if torch.is_tensor(waveform):
        waveform = waveform.detach().float().cpu().numpy()
    sf.write(Path(output_path), waveform.squeeze(), 16_000)


# Text-to-speech
result = engine.generate_tts(
    sentence="FlexiSLM supports controllable speech generation.",
    framerate=8.0,
)
save_audio(result, "tts.wav")

# ASR
result = engine.generate_from_audio(
    audio_path="examples/input.wav",
    text_query="Please transcribe the audio.",
    framerate=8.0,
    output_text_only=True,
)
print(result["text"])

# Audio QA
result = engine.generate_from_audio(
    audio_path="examples/question.wav",
    text_query="",
    framerate=8.0,
    output_text_only=True,
)
print(result["text"])

# Speech-to-speech
result = engine.generate_from_audio(
    audio_path="examples/input.wav",
    text_query="",
    framerate=8.0,
    output_text_only=False,
)
save_audio(result, "s2s.wav")

Manual download

MODEL_ROOT="$PWD/models"

hf download FlexiSLM/FlexiSLM-0_5B-Stage2 --local-dir "$MODEL_ROOT/FlexiSLM-0_5B-Stage2"

# Shared auxiliary files (required for inference; same as 7B)
hf download FlexiSLM/Qwen2_5-Omni-Audio_Encoder --local-dir "$MODEL_ROOT/Qwen2_5-Omni-Audio_Encoder"
hf download FunAudioLLM/SenseVoiceSmall --local-dir "$MODEL_ROOT/SenseVoiceSmall"
hf download jiaqili3/flexicodec \
  12hz_v1_half_config.yaml \
  nartts_flexicodec_only.safetensors \
  nartts.safetensors \
  --local-dir "$MODEL_ROOT/FlexiCodec"
hf download amphion/dualcodec-tts vocos_emilia.safetensors \
  --local-dir "$MODEL_ROOT/FlexiCodec"

Then point FlexiSLMInferenceConfig at the local paths (checkpoint="stage2_0.5B", model_path=models/FlexiSLM-0_5B-Stage2, plus encoder/codec paths). See the code README Inference section for the full config block.

Batch inference

Reuse examples/infer_7b.yaml with the 0.5B checkpoint:

engine:
  config:
    checkpoint: stage2_0.5B
    model_path: models/FlexiSLM-0_5B-Stage2
    # ... shared encoder / FlexiCodec / SenseVoice paths ...
inference:
  checkpoint: models/FlexiSLM-0_5B-Stage2

python -m src.infer examples/infer_7b.yaml

Or set engine.config.auto_download: true with checkpoint: stage2_0.5B.

Controllable frame rate

A single Stage 2 model can be steered between about 12.5 Hz and 4.0 Hz without retraining via input_framerate / default_framerate / per-call framerate.

Citation

@misc{li2026flexislmdynamiccontrollableframe,
      title={FlexiSLM: A Dynamic and Controllable Frame Rate Spoken Language Model},
      author={Jiaqi Li and Chaoren Wang and Xiaohai Tian and Mingjie Chen and Xinyu Liang and Xu Li and Yufan Lin and Junwen Qiu and Jun Zhang and Lu Lu and Haizhou Li and Zhizheng Wu},
      year={2026},
      eprint={2606.31247},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2606.31247},
}