SeeQ-3B

SeeQ-3B

SeeQ: Training Generalist Value Functions for Long-Horizon Robotic Manipulation

Saksham Singh, Zheyuan Hu, Max Sobol Mark, Jeffrey Yu, Zackory Erickson, Aviral Kumar (Carnegie Mellon University)

Project website (with videos): https://saksham002.github.io/seeq/

Paper: https://arxiv.org/abs/2609.22085

Model description

SeeQ (Subtask-elicited Q-functions) is a generalist Q-value function for language-conditioned robot manipulation. Rather than modeling sparse success over the full task, SeeQ estimates the expected return for the currently active subtask, which shortens the value-prediction horizon and makes temporal-difference (TD) learning effective on long-horizon tasks. At inference time the model first autoregressively decodes the active subtask in natural language, then scores candidate action chunks conditioned on it. The resulting Q-values are used for best-of-N steering of a base policy (pi-0.5 in the paper).

This repository contains the RoboCOIN-pretrained generalist checkpoint (230k training steps). In the paper this checkpoint is fine-tuned on each downstream task before deployment.

Architecture

• Backbone: PaliGemma 3B (SigLIP image encoder + Gemma language model), initialized from the base PaliGemma weights.
• Inputs: 3 camera images (224x224), the task instruction, the active subtask text, and a 50-step, 14-dim bimanual action chunk (linearly projected into action tokens). Proprioceptive state is not an input.
• Image and task tokens attend bidirectionally. Subtask tokens are causal (trained with a next-token loss) and attend to the images, the task and preceding subtask text. Action tokens attend bidirectionally within the chunk and to all preceding image and language tokens.
• A learned value token at the end of the sequence attends to all valid preceding tokens; a linear readout maps it to a scalar Q-value.

Training

Training config: robocoin_bimanual_paligemma_cql_rlds_subtask_ar.

Files

This is an openpi-style Orbax checkpoint (JAX / Flax NNX):

• params/ — model parameters (Orbax PyTree checkpoint). Optimizer state is not included.
• assets/embodiment_wise/norm_stats.json — normalization statistics used during training; inputs must be normalized with these at inference time.
• _CHECKPOINT_METADATA — Orbax checkpoint metadata.

Intended use and limitations

• Intended as an initialization for fine-tuning a Q-function on a downstream bimanual manipulation task, and for research on value learning and policy steering.
• Values are subtask-level returns under the RoboCOIN data distribution; they are not calibrated task-success probabilities. Out of the box, results on new embodiments, camera setups or action spaces are not guaranteed; the paper fine-tunes on target-task data before deployment.
• This model scores actions; it does not generate them. It must be paired with a base policy that proposes candidate action chunks.

Simulation vs. real-world deployment

This checkpoint is pretrained entirely on real-robot data, so its value estimates are tuned to real-world dynamics and visuals. On the simulation tasks we have tested, fine-tuning it yielded little improvement over the base policy, both with RaC (Recovery and Correction) data and with purely teleoperated data. We hypothesize that this is a property of the pretraining mixture rather than of the method: Section 4.3 of the paper shows that broad robot-action pretraining is what enables SeeQ to transfer to a new task, and simulated dynamics and rendering differ enough from real-world data that this transfer does not carry over to simulation.

We therefore recommend using this checkpoint for real-world tasks, where its pretraining is in-distribution and where the paper reports the largest gains. Real-world performance is ultimately what the method targets. If your goal is a simulation benchmark, the SeeQ objective is not tied to real data: pretrain a SeeQ model on a broad simulation dataset with the training config above, then fine-tune on the target task.

Citation

@article{singh2026seeq,
  title   = {SeeQ: Training Generalist Value Functions for Long-Horizon Robotic Manipulation},
  author  = {Singh, Saksham and Hu, Zheyuan and Mark, Max Sobol and Yu, Jeffrey and Erickson, Zackory and Kumar, Aviral},
  journal = {arXiv preprint arXiv:2609.22085},
  year    = {2026},
  url     = {https://arxiv.org/abs/2609.22085}
}

License

The weights are derived from PaliGemma and are subject to the Gemma Terms of Use.