svp-v-coconut-gpt2

SVP-V-GRPO · COCONUT GPT-2

A COCONUT GPT-2 (124M) latent-reasoning model post-trained with SVP-V-GRPO — reinforcement learning whose rollout exploration comes from weight-space perturbation of the attention value projections. For each layer and rollout, SVP independently samples Gaussian noise and rescales coefficients in the fixed SVD basis of W_V as sᵢ → sᵢ(1+αgᵢ).

Only the value-projection weights (the V columns of each c_attn) differ from the base checkpoint. The perturbation is a training-time exploration mechanism and is not used at deployment: inference is ordinary greedy decoding. This repository contains the paper's GPT-2 SVP-V-GRPO checkpoint from epoch 9 B=32, lr=6e-5 run.

Results

Clean greedy decoding, max_new_tokens=64, exact-match on the final answer.

Usage

This is a fixed-length continuous-thought model: the prompt ends with <|start-latent|>, six latent steps feed each step's last hidden state back as the next input embedding, <|end-latent|> closes the latent phase, and the answer is then decoded as ordinary tokens.

from transformers import AutoTokenizer, GPT2LMHeadModel

REPO = "."  # path to this repository
tok = AutoTokenizer.from_pretrained(REPO)
model = GPT2LMHeadModel.from_pretrained(REPO).eval().cuda()
# then run the two-phase latent loop: 6 latent steps, then decode the answer

A reference implementation of the loop ships with the SVP code release; the original COCONUT inference code works as well.

Training

License

MIT, inherited from the base checkpoint (which derives from openai-community/gpt2).