StateDiffRWKV-2.9B-512-pretrained

DiffRWKV State-Hijacking RELAY 2.9B 512 Trajectory Champion (from-scratch joint co-adapt + condboundary, pretrained/no-SFT, 57.67)

This private release contains a DiffRWKV State-Hijacking RELAY checkpoint plus the minimal inference code needed to load it.

Files

• model.pt: trainable RELAY checkpoint only. It does not include the frozen RWKV backbone weights.
• configs/rwkv_relay_2.9B_state_hijack_dit_vae32_traj32x16.yaml: training/inference config used by the checkpoint.
• models/state_hijacking_dit.py: RELAY model implementation.
• scripts/eval/relay_utils.py: checkpoint loader.
• scripts/eval/sample_prefix_suffix_trajectory_cfg.py: inference entrypoint.
• manifest.json: size, SHA256, source path, and step metadata.

Base Model

RWKV/RWKV7-Goose-World3-2.9B-HF

The loader expects the base RWKV path in the checkpoint config to exist locally. If your path differs, edit cfg.model.rwkv_local_path in scripts/eval/relay_utils.py after loading or patch the config before constructing the model.

Install

pip install -r requirements.txt
pip install flash-linear-attention fla-core

Use the same FLA build as the training environment when reporting numbers.

Inference

CUDA_VISIBLE_DEVICES=0 python scripts/eval/sample_prefix_suffix_trajectory_cfg.py --ckpt_dir . --prompt 'The history of artificial intelligence' --output sample.json --steps 100 --cfg_scale 3 --max_new_tokens 512 --trajectory_s1_mode independent --trajectory_state_blend 0.7 --temperature 0.5 --top_k 5 --top_p 0.7 --repetition_penalty 1.2

Upload

huggingface-cli repo create SII-Jiaquan/StateDiffRWKV-2.9B-512-pretrained --type model --private
huggingface-cli upload SII-Jiaquan/StateDiffRWKV-2.9B-512-pretrained . . --repo-type model