gslm-scaling-823m-82b

gslm-scaling-823m-82b

Model-only FP32 weights for a complete endpoint from Scaling Properties of Speech Language Models (EMNLP 2024). The model predicts consecutive-run-collapsed, 25 Hz layer-11 mHuBERT K-means units without waveform normalization (K=500; EOS=500; PAD=501). A vocoder is not bundled.

Speech-tokenizer and vocoder artifacts are credited in the SLM third-party notices.

import torch
from slm import TransformerLM

model = TransformerLM.from_pretrained("tiagoCuervo/gslm-scaling-823m-82b", device="cuda")
prompt = torch.tensor([[12, 91, 204]], device="cuda")
units = model.generate(prompt, max_new_tokens=200, temperature=0.8, top_k=50)

BF16 evaluation scores are 61.33% sBLIMP phenomenon macro accuracy, 58.36% sBLIMP voice-pair accuracy, 56.92% sStoryCloze, and 77.93% tStoryCloze. See provenance.json for hashes and exact validation metadata.

Use the slm-research codebase for data preparation, evaluation, and audio decoding. Source datasets and the CodeHiFiGAN vocoder retain their own licenses and are not included.