SDRPN-Gemma-4-12B

YuhengSSS/SDRPN-Gemma-4-12B

Papers: https://arxiv.org/abs/2609.19745 (Vision-RL²) · https://arxiv.org/abs/2509.16944 (SD-RPN)

SD-RPN stage-1 checkpoint: a self-distilled RoI predictor twig (K = 27, T = 3) trained on a frozen, encoder-free google/gemma-4-12B-it. This is the initialisation of the Vision-RL² RL run YuhengSSS/VisionRL2-Gemma-4-12B.

The backbone weights are unchanged from the base model; only the three attached twig blocks are trained, from self-distilled attention pseudo-labels (no human RoI annotation).

Repository contents

The Gemma stage-1 driver trains the delta and its assemble stage turns it into the full directory; both forms are published so you can start from either.

Configuration

How to use

These weights need the modeling code in YuHengsss/VisionRL2. They are not loadable for RoI inference through a plain AutoModel / AutoModelForCausalLM call: the RoI gating path (heatmap head, peak-relative gate, connected-component crop, sub-image splice) lives in the repository's model classes and evaluation harness.

git clone https://github.com/YuHengsss/VisionRL2
cd VisionRL2
hf download YuhengSSS/SDRPN-Gemma-4-12B --local-dir output/sdrpn/gemma4-12b-sdrpn-K27T3

# stage 2: region-level RL starting from this checkpoint
PHASE_A_CKPT=output/sdrpn/gemma4-12b-sdrpn-K27T3 \
  DATASET_ROOT=datasets GPU_IDS=0,1 bash scripts/train_rl_gemma4_12b.sh

# evaluate the stage-1 checkpoint itself (training-aligned protocol, dense crop)
CHECKPOINT=output/sdrpn/gemma4-12b-sdrpn-K27T3 ROI_MODE=dense bash scripts/aligned_eval_gemma4.sh

To start from the delta instead, download only twig_delta/twig_delta_final.pt and run qwen_src/gemma4_unified/assemble_full_checkpoint.py against the base model (see docs/GEMMA4.md).

Expected numbers (sanity check after download)

Training-aligned protocol (rule metrics, no judge), source tier 1120 - the SD-RPN (stage 1), dense crop row of the Vision-RL² README:

Links

• Code: https://github.com/YuHengsss/VisionRL2
• Data (SD-RPN corpora, RL pools, evidence maps): https://huggingface.co/datasets/YuhengSSS/VisionRL2-data
• Collection: https://huggingface.co/collections/YuhengSSS/visionrl2
• Project page: https://yuhengsss.github.io/VisionRL2/

Citation

@article{shi2026visionrl2,
  title   = {Region-Level Policy Optimization for Fine-grained MLLM Perception},
  author  = {Shi, Yuheng and Pei, Xiaohuan and Dong, Minjing and Xu, Chang},
  journal = {arXiv preprint arXiv:2609.19745},
  year    = {2026}
}

@inproceedings{shi2026sdrpn,
  title     = {Catching the Details: Self-Distilled RoI Predictors for Fine-Grained MLLM Perception},
  author    = {Shi, Yuheng and Pei, Xiaohuan and Dong, Minjing and Xu, Chang},
  booktitle = {ICLR},
  year      = {2026}
}