gsm-cot-llama3b

GSM8K CoT fine-tune of Llama-3.2-3B-Instruct

Built with Llama. Supervised chain-of-thought fine-tune of meta-llama/Llama-3.2-3B-Instruct on GSM8K.

This model serves as the Stage-1 CoT baseline/initialization checkpoint for LOTUS (Looped Transformers with parallel supervision on latents), as described in the paper Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers.

Links

• Paper: arXiv:2606.31779
• Code: GitHub - yingfan-bot/lotus

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("yingfanbot/gsm-cot-llama3b")
tokenizer = AutoTokenizer.from_pretrained("yingfanbot/gsm-cot-llama3b")

Citation

@misc{fan2026bridginggaplatentexplicit,
      title={Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers}, 
      author={Ying Fan and Anej Svete and Kangwook Lee},
      year={2026},
      eprint={2606.31779},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2606.31779}, 
}