gsm-cot-llama1b

GSM8K CoT fine-tune of Llama-3.2-1B-Instruct

Built with Llama. Supervised chain-of-thought fine-tune of meta-llama/Llama-3.2-1B-Instruct on GSM8K.

This model serves as the Stage 1 supervised fine-tuning (SFT) baseline for LOTUS (Looped Transformers with parallel supervision on latents), as presented in the paper Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers.

• Paper: Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers
• Repository: GitHub - yingfan-bot/lotus

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("yingfanbot/gsm-cot-llama1b")
tokenizer = AutoTokenizer.from_pretrained("yingfanbot/gsm-cot-llama1b")

Citation

If you use this model or code, please cite the paper:

@misc{fan2026bridginggaplatentexplicit,
      title={Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers}, 
      author={Ying Fan and Anej Svete and Kangwook Lee},
      year={2026},
      eprint={2606.31779},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2606.31779}, 
}