R2-dLLM-Dream

$R^2$-dLLM-Dream

This repository contains the $R^2$-dLLM-Dream model, an accelerated Diffusion Large Language Model (dLLM) introduced in the paper $R^2$-dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reduction.

$R^2$-dLLM is a unified framework for reducing decoding redundancy in dLLMs from both inference and training perspectives. This specific checkpoint is a redundancy-aware supervised fine-tuned (SFT) version based on Dream-v0-Instruct-7B.

• Code: GATECH-EIC/R2-dLLM
• Paper: arXiv:2604.18995

Summary

Diffusion Large Language Models (dLLMs) enable parallel token prediction but often suffer from high inference latency due to recurring spatial and temporal redundancy during decoding. $R^2$-dLLM addresses this through:

1. Inference-time decoding rules: Training-free rules that aggregate local confidence and finalize temporally stable tokens to avoid redundant decoding steps.
2. Redundancy-aware SFT: A fine-tuning pipeline that aligns the model with efficient decoding trajectories.

Experiments demonstrate that $R^2$-dLLM can reduce the number of decoding steps by up to 88% compared to existing strategies while maintaining competitive generation quality.

Citation

If you find this work useful for your research, please cite:

@article{du2026r,
  title={$R^{2}$-dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reduction},
  author={Du, Zhenbang and Xia, Kejing and Zhong, Xinrui and Fu, Yonggan and Oswald, Nicolai and Ji, Binfei and Khailany, Brucek and Molchanov, Pavlo and Lin, Yingyan},
  journal={arXiv preprint arXiv:2604.18995},
  year={2026}
}