gsm-cot-gpt2

LOTUS (Looped Transformers with parallel supervision on latents)

This model checkpoint is part of the project LOTUS, introduced in the paper Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers.

LOTUS is a latent Chain-of-Thought (CoT) reasoning method designed for looped Transformers. It processes reasoning steps in parallel in the model's hidden states, allowing for significantly lower latency than explicit token-by-token generation while bridging the performance gap to explicit CoT.

Model Details

• Developed by: Ying Fan, Anej Svete, Kangwook Lee
• License: MIT
• Base Model: GPT-2

Resources

• Repository: GitHub - yingfan-bot/lotus
• Paper: Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers

Evaluation

To evaluate this checkpoint on the GSM8K dataset, you can run the evaluation script from the official repository:

python scripts/eval.py \
  --checkpoint yingfanbot/gsm-lotus-gpt2 \
  --model_id gpt2 \
  --datasets gsm8k \
  --n_looped_iters 6 --c_thought 13

For environment setup, training details, and further evaluation instructions, please refer to the official GitHub Repository.

Citation

@misc{fan2026bridginggaplatentexplicit,
      title={Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers}, 
      author={Ying Fan and Anej Svete and Kangwook Lee},
      year={2026},
      eprint={2606.31779},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2606.31779}, 
}