Diffusion-Learning-via-Strategic-Denoising-and-Remasking

DRPO 1.7B

This repository provides the final tensor checkpoint for DRPO 1.7B, trained with strategic denoising and remasking. It is intended to be used with the canonical training and Hugging Face evaluation code in the DRPO reproduction repository.

Model details

Reproduce the evaluation

The reproduction repository provides a one-command environment setup, pinned model/data revisions, the canonical model implementation, and the exact MATH-500 recipe.

git clone https://github.com/leotsia0416/DRPO.git
cd DRPO
python3 setup.py
sbatch script/test_gap.sh math500

The evaluation parameters are recorded in script/config/eval_math500.yaml. setup.py automatically downloads this checkpoint and the required base-model assets.

Checkpoint contents

Only the final model tensors and safetensors index are published here:

• model-00001-of-00002.safetensors
• model-00002-of-00002.safetensors
• model.safetensors.index.json

Executable model code, tokenizer assets, optimizer/scheduler states, RNG states, and checkpoint-local source snapshots are intentionally excluded. This prevents stale checkpoint code from overriding the canonical implementation in Git.

Reproducibility note

The archived evaluation predictions score 322/500 (64.4%), and the reproduction repository pins the same MATH answer-extraction rule. Generated text can still vary across GPU jobs because BF16 kernels and FlashAttention are not guaranteed to be bit-identical across nodes, drivers, and CUDA stacks, even with temperature: 0.0.

File integrity (SHA-256)

caa92001f2b4b62433db1b8b16702ba35726b07b1442cf814bdfd4b1951fee0f  model-00001-of-00002.safetensors
e5a3575ce8b1481d7ba949c841c37edc7c13f70262b69b29164c85d37a33441e  model-00002-of-00002.safetensors
cb0bbd72bc89e405e31718fe07d9d1fb560faf4795d64811e572fa7216d085bb  model.safetensors.index.json