VLAct-Qwen3vl4b-OFT-RoboDojo

VLAct QwenOFT Qwen3-VL-4B for RoboDojo (100K Fine-tuned)

This repository contains a RoboDojo-fine-tuned StarVLA QwenOFT policy. It was fine-tuned for 100,000 steps on the 35-task RoboDojo LeRobot v2.1 mixture using 2 nodes x 8 GPUs. The uploaded checkpoint is the exact steps_100000_pytorch_model.pt used for the evaluation reported below. It is a complete framework state dict, not a LoRA or adapter-only release.

Initialization

VLAct pretrained describes only the initialization source. RoboDojo fine-tuning started from the 100K-step 0706 multi-embodiment, three-head VLAct pretraining run. The checkpoint released here is the resulting RoboDojo fine-tune; it is not the raw VLAct pretraining checkpoint.

Model details

The original run used the action-extended Qwen3-VL snapshot at revision 8c99a9a3db659019e6e785add928fcea8e14a18a. The packaged config.yaml uses the public Hub model ID so that the framework can reconstruct the model. training_config.original.yaml preserves the exact training configuration and local provenance paths used by the run.

Training details

RoboDojo results

Values are shown as success rate (SR, %) and score; higher is better. Group values are unweighted means over their tasks, and Average is the unweighted mean over all 42 tasks.

Evaluation protocol

• 42 tasks, seeds 0/1/2, 10 episodes per task and seed: 1,260 episodes total.
• Each task therefore has 30 evaluated episodes.
• Each Generalization task uses 5 standard and 5 randomized episodes per seed, preserving the official standard/random split at one-fifth scale.
• All other tasks use 10 episodes per seed.
• Evaluation uses state (include_state=true), arx_x5 normalization, three camera inputs, and executes 16 actions before requesting the next chunk.
• Task SR is successful episodes divided by 30. Task score is the mean episode score multiplied by 100.

This is a scaled local reproduction protocol. It is not the official 50-episode-per-task / 2,100-episode protocol, and the values should not be presented as a verified official leaderboard submission.

Per-task results

Machine-readable values and protocol metadata are in evaluation_results.json.

Files

README.md
config.yaml
dataset_statistics.json
evaluation_results.json
summary.jsonl
training_config.original.yaml
checkpoints/
└── steps_100000_pytorch_model.pt

config.yaml and dataset_statistics.json must remain two directory levels above the checkpoint file; StarVLA resolves both relative to the checkpoint.

Inference

Download the repository, then start the StarVLA model server:

huggingface-cli download JasonYang66/VLAct-Qwen3vl4b-OFT-RoboDojo \
  --local-dir VLAct-Qwen3vl4b-OFT-RoboDojo

CKPT=VLAct-Qwen3vl4b-OFT-RoboDojo/checkpoints/steps_100000_pytorch_model.pt
python deployment/model_server/server_policy.py \
  --ckpt_path "$CKPT" \
  --port 57700 \
  --use_bf16

RoboDojo evaluation must match the training-time input contract:

export STARVLA_CKPT_PATH="$CKPT"
export STARVLA_INCLUDE_STATE=True
export STARVLA_UNNORM_KEY=arx_x5
export STARVLA_EXECUTE_HORIZON=16

Changing state usage, camera ordering, normalization statistics, action order, or execution horizon can materially change the result.

Intended use

This checkpoint is intended for research on RoboDojo with the ARX X5 dual-arm embodiment. Performance under different camera calibration, state/action ordering, normalization, robot embodiments, or real hardware has not been established.