FlexiSLM-0_5B-Stage1

FlexiSLM-0.5B-Stage1

Stage 1 (talker / input-module pre-training) checkpoint for FlexiSLM-0.5B.

This checkpoint is the default initialization for Stage 2 multi-task LoRA fine-tuning. For end-user inference (TTS / ASR / audio QA / speech-to-speech), use the Stage 2 release instead: FlexiSLM/FlexiSLM-0_5B-Stage2.

• Paper: arXiv:2606.31247
• Demo: flexislm.github.io
• Code: AmphionTeam/FlexiSLM
• Sibling: FlexiSLM-7B-Stage1

This project is in active development. Checkpoints may be overwritten as training continues.

Training stages (reminder)

1. Stage 1 (this repo): freeze the Qwen backbone; train Talker, audio embeddings, and input frame-merging.
2. Stage 2: train Talker + input modules; adapt Thinker with LoRA → FlexiSLM-0_5B-Stage2.
3. Stage 3: merge Stage 2 LoRA, enable Talker→Thinker connection, full fine-tune.

Use as Stage 2 initialization

Install the code and download shared assets as in the Training Guide.

git clone --recurse-submodules https://github.com/AmphionTeam/FlexiSLM.git
cd FlexiSLM
pip install -r requirements.txt

MODEL_ROOT="$PWD/models"

# Optional local mirror (Stage 2 can also pull the Hub id automatically)
hf download FlexiSLM/FlexiSLM-0_5B-Stage1 --local-dir "$MODEL_ROOT/FlexiSLM-0_5B-Stage1"

# Shared encoder / codec + Qwen backbone (see repo README for full list)
hf download FlexiSLM/Qwen2_5-Omni-Audio_Encoder --local-dir "$MODEL_ROOT/Qwen2_5-Omni-Audio_Encoder"
hf download FunAudioLLM/SenseVoiceSmall --local-dir "$MODEL_ROOT/SenseVoiceSmall"
hf download jiaqili3/flexicodec \
  12hz_v1_half_config.yaml \
  nartts_flexicodec_only.safetensors \
  --local-dir "$MODEL_ROOT/FlexiCodec"
hf download Qwen/Qwen2.5-0.5B-Instruct --local-dir "$MODEL_ROOT/Qwen2.5-0.5B-Instruct"

config/train_stage2_0_5B.yaml already sets:

resume_from_checkpoint: FlexiSLM/FlexiSLM-0_5B-Stage1

Launch Stage 2:

bash scripts/train_stage2_0_5B.sh

Override on the command line if needed:

bash scripts/train_stage2_0_5B.sh \
  --resume_from_checkpoint FlexiSLM/FlexiSLM-0_5B-Stage1 \
  --output_dir outputs/train_stage2_0_5B

Hub ids such as FlexiSLM/FlexiSLM-0_5B-Stage1 are downloaded into models/ when missing.

Inference

Prefer FlexiSLM-0_5B-Stage2 with checkpoint="stage2_0.5B" (auto-download or manual). Full Python / batch examples are in the code README.

Citation

@misc{li2026flexislmdynamiccontrollableframe,
      title={FlexiSLM: A Dynamic and Controllable Frame Rate Spoken Language Model},
      author={Jiaqi Li and Chaoren Wang and Xiaohai Tian and Mingjie Chen and Xinyu Liang and Xu Li and Yufan Lin and Junwen Qiu and Jun Zhang and Lu Lu and Haizhou Li and Zhizheng Wu},
      year={2026},
      eprint={2606.31247},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2606.31247},
}