RA-FM

RA-FM Scale-L checkpoints

These are the three inference-only checkpoints for the primary RA-FM K=2 configuration from Rollout Alignment for Flow-Based Speech Continuation. Each model was fine-tuned for 20,000 updates on the audited LibriSpeech train-clean-100 Scale-L subset with a different training seed.

Files

model.safetensors contains the 292 tensors in model_trainable_state. It does not contain optimizer state, scheduler state, data, or the frozen codec. Each checkpoint must be applied to the Flow-SLM-1B-Extended base model.

Code and loading

The exact architecture, training, evaluation, and Slurm reproduction code is at JoyBoyuuu/RA-FM. Download one seed and load its tensors into the upstream model before inference:

import torch
from safetensors.torch import load_file
from tools.training.train_catcfm_family_a import load_model

# Run from a JoyBoyuuu/RA-FM checkout. Download the original upstream base
# checkpoint separately; it is not redistributed in this model repository.
model, config = load_model(
    "checkpoints/flow_slm_1b_extended.bin",
    "conf/1b_extended.yaml",
)
state = load_file("seed_42/model.safetensors", device="cpu")
missing, unexpected = model.load_state_dict(state, strict=False)
assert not unexpected
model = model.to("cuda").eval()
for parameter in model.parameters():
    parameter.requires_grad = False

The missing keys are expected: the files contain the full set of parameters that were trainable during RA-FM fine-tuning, while frozen base-model/codec parameters come from Flow-SLM-1B-Extended. The base checkpoint used in the experiments had SHA-256:

73bb302b6639bcef25260393e16848e22d83225088cae8bf0096499fcdb41714

See the repository's reproducibility guide for data preparation, exact inference settings, and evaluation commands.

Training objective

RA-FM retains conditional flow matching and the inherited pointwise speech-time terms, then differentiates an auxiliary speech-dynamics objective through the model's own fixed-step Euler rollout. The primary model uses K=2 training rollout steps, temperature 0.8, CFG scale 0.3, and a rollout weight calibrated to a 0.10 rollout-to-CFM FlowHead gradient-norm ratio.

Results and limitations

Against matched FULL-TCFM seeds, K=2 improved continuous rollout trajectory error by 1.45% and hard-RVQ trajectory error by 1.27%. In controlled decoded evaluation, all seeds improved motion (0.57--0.60%), variation (0.40--0.43%), energy movement (1.08--1.15%), and boundary error (0.18--0.26%), while WavLM speaker similarity was nearly neutral (-0.10% to +0.00%).

The evaluation uses teacher-forced causal context and oracle future semantic tokens. These checkpoints therefore support the claim of improved decoded temporal dynamics under oracle semantic continuation; they are not a fully free-running solution to autoregressive exposure bias.

License and attribution

These weights are derived from Flow-SLM. The upstream repository did not provide a license when these artifacts were prepared, so this repository uses license: other and does not imply additional redistribution or commercial-use rights. Users must comply with the terms of Flow-SLM, its base checkpoint, Mimi, LibriSpeech, and all dependencies.

Citation

@article{chou2025flow,
  title={Flow-SLM: Joint Learning of Linguistic and Acoustic Information for Spoken Language Modeling},
  author={Chou, Ju-Chieh and Zhou, Jiawei and Livescu, Karen},
  journal={arXiv preprint arXiv:2508.09350},
  year={2025}
}

The RA-FM paper citation will be added after publication.