smolvla-libero-spatial-4arm

SmolVLA on LIBERO-Plus libero_spatial — four post-training arms

This repository contains four SmolVLA checkpoints from a single-GPU proof-of-concept study that compared four post-training recipes on the libero_spatial suite of LIBERO-Plus, all starting from the same supervised fine-tune:

Read this before you download anything

These weights are published so that a third party can re-run the reported evaluations. They are not published because any arm is good, and this card does not recommend any arm over any other. Success rates are low in absolute terms (13–43% depending on the evaluation), the study is n=1 per arm, and the two findings below are the honest summary of what the comparison established.

1. GRPO is not better than the checkpoint it started from

The study's only confirmatory result was preregistered — design, primary statistic and decision rule were all committed before a single episode ran. It contrasted grpo/ against its own initialization sft/ on tasks that were never trained on:

• Primary statistic: mean per-task success-rate difference (GRPO − SFT) = +0.72 pp, 95% task-clustered bootstrap CI [−5.07, +6.52] pp (20 000 resamples). The interval includes zero.
• Secondary: exact McNemar on discordant episode pairs (8 GRPO wins, 7 SFT wins) → p = 1.0.
• Under the decision rule fixed in advance this is reported as NOT ESTABLISHED.

Do not choose grpo/ over sft/ expecting better performance. There is no established advantage. A null is also not proof of equivalence or inferiority — the interval rules out an effect as large as an earlier exploratory probe had suggested, and nothing more.

2. rs_sft/ and dpo/ carry a known bias — the trained-task table flatters them

Both rs_sft/ and dpo/ were trained on rollouts collected at the very initial states that the trained-task evaluation (eval A) then used.

LIBERO-Plus does not let you choose the initial state with the reset seed: LiberoEnv.reset() applies the seed and then overwrites the state with set_init_state(init_states[init_state_id % N]), where init_state_id starts from the sub-environment's episode_index and advances by n_envs on every reset. Eval A was run with a seed chosen to be disjoint from all training seeds, in the belief that this produced held-out initial conditions. It did not: eval A ran on training initial states.

This matters asymmetrically. RS-SFT behaviour-clones successful trajectories from exactly those states and flow-DPO's preference pairs come from the same collection, so eval A favours these two arms by construction. Adding episodes would not remove the bias. Two of the eight training tasks (1817, 1955) have len(init_states) == 1, so collection, RL training and every eval-A episode on them began from the identical state.

RS-SFT topping the eval A table is exactly what the bias predicts, and cannot be distinguished from a real effect. Anyone who reads only the eval A numbers will conclude RS-SFT is the best arm. That conclusion is not supported.

Restricted to the genuinely paired subset — 56 of the 105 episodes per arm, see below — RS-SFT leads SFT by +12.5 pp but the contrast is inconclusive: exact McNemar p = 0.092, task-clustered CI [−3.6, +30.4] pp.

3. The paper is a preprint only, and this card does not pretend otherwise

The manuscript is a preprint (10.5281/zenodo.21596932; v1 is 10.5281/zenodo.21596933). It has not been published or accepted anywhere and no venue has been selected, so do not cite it as a published paper. Every claim on this card is also backed by a file in the public repository, so you can check any of it without reading the preprint.

Contents, sizes and hashes

Each subfolder is a complete LeRobot save_pretrained directory (config.json, model.safetensors, and the pre/post-processor files). SHA-256 is over the file bytes; model.safetensors holds the weights. sft/ carries one extra file, train_config.json — the full LeRobot training configuration of the SFT run (dataset, policy, optimizer, scheduler, steps, seed), kept for reproducibility.

No optimizer state (optim.pt) is published, so these checkpoints are for inference and evaluation, not for resuming training.

The folder content hash is sha256 over the sorted "<filename> <sha256>\n" lines of the files listed above, so it identifies the exact set of bytes published in that subfolder. Reproduce it with:

for f in $(ls | sort); do printf '%s %s\n' "$f" "$(sha256sum "$f" | cut -d' ' -f1)"; done | sha256sum

Source checkpoint directories on the author's Drive (for the record):

• sft/ ← MyDrive/SmolVLA_RL/outputs/sft_base_heldout/checkpoints/020000/pretrained_model
• rs_sft/ ← MyDrive/SmolVLA_RL/rs_sft_run/checkpoints/last/pretrained_model
• dpo/ ← MyDrive/SmolVLA_RL/dpo_run/checkpoints/last/pretrained_model
• grpo/ ← MyDrive/SmolVLA_RL/grpo_run8/ckpt_latest/pretrained_model

Provenance of each arm

Common ancestor: lerobot/smolvla_base (~450M params: SmolVLM2 vision-language backbone + flow-matching action expert).

lerobot/smolvla_base
  └─ sft/      SFT on lerobot/libero_plus, ALL camera_pose categories held out
       ├─ rs_sft/   rejection-sampling SFT   ┐ both trained on the SAME 256-episode
       ├─ dpo/      flow-DPO                 ┘ rollout collection from sft/
       └─ grpo/     Flow-SDE GRPO (run8 final, 20 iterations)

Training tasks (LeRobot 0-based ids, libero_spatial): 79, 108, 1477, 1530, 1817, 1955, 2126, 2172 — selected by the reproducible rule in src/select_grpo_tasks.py, not by hand.

GRPO's reward is a monotonically gated staged privileged reward read from simulator state (0.15 approach + 0.20 reached + 0.20 grasp_attempted + 0.20 lifted + 0.25 success) with a target-identity lock. It does not transfer to settings without privileged state.

Evaluations in full

Confirmatory: preregistered unseen-task test (grpo/ vs sft/ only)

48 unseen libero_spatial tasks were drawn; 2 errored for both arms and were dropped under the prespecified rule, leaving 46 effective tasks × 3 paired initial states = 138 episodes per arm. Both arms saw identical initial-state sets per task (asserted per episode). The vector environment is rebuilt per wave so the initial state is a function of (offset, wave, batch) only, never of an arm's own success pattern. Result: the table in §1 — NOT ESTABLISHED.

rs_sft/ and dpo/ were not run in this evaluation. There is therefore no unconfounded head-to-head number for them anywhere in this study.

• Preregistration: data/eval/PREREGISTRATION_unseen_task_eval.md (includes Amendment 1)
• Result: data/eval/eval_unseen_prereg_result.json
• Raw per-episode outputs: data/eval/raw/sft_result.json, data/eval/raw/grpo_result.json
• Recompute every number, including the secondary endpoint:

  python analysis/analyze_unseen_prereg.py \
      data/eval/raw/sft_result.json data/eval/raw/grpo_result.json
  

Trained-task evaluation (eval A) — confounded, not a ranking

Four arms on the same 7 trained tasks (task 1477 excluded: deterministic LIBERO-Plus renderer shape bug for all arms), 15 episodes/task = 105 episodes per arm. Read §2 above before reading this table.

Two separate defects, both consequences of the same mechanism:

1. Bias — the initial states were the training ones, favouring rs_sft/ and dpo/ (§2).
2. Broken pairing — a terminated episode triggers at least two resets (LiberoEnv.step's internal reset plus gymnasium's NEXT_STEP autoreset — at least, because the slot keeps being stepped afterwards and can terminate again), so from the second wave onward each arm's initial states depended on its own outcome pattern. Only 56 of 105 episodes per arm (wave 0) are genuinely paired. That restriction is unaffected by the correction below: it holds because wave 0 precedes any outcome-dependent history, which is exactly why we restrict rather than try to reconstruct.

On the genuinely paired wave-0 subset:

Δ = +12.5 pp, exact McNemar p = 0.092, task-clustered CI [−3.6, +30.4] pp → not significant. The point estimate is unchanged, but the significance reported over the full 105 episodes rests substantially on wave 1, where the pairing assumption fails (wave 1 alone: p = 0.0215 — reported only to show where the full-sample significance came from). This re-analysis was done for rs_sft/ vs sft/; it was not extended to the other arms.

An offset cannot repair eval A: because a terminated episode causes extra resets, collection touches a scattered, outcome-dependent set of indices — and we cannot say which. src/init_state_coverage.py estimated that it reaches as high as 79 and, wrapping via % N at the real N = 50, blocks 32 of 50 states leaving no free window of the required width 15.

Correction (2026-07-29). Those figures are an estimate, not an established fact, and this card previously presented them as fact. The reconstruction assumes a terminated episode costs exactly one internal reset plus one autoreset. It can be wrong in both directions: a done slot is not frozen — the collection loop runs until every slot is done, so it keeps being stepped, runs a fresh episode, and can terminate again (undercount) — while a termination on the step that ends the loop never gets its autoreset (overcount). terminated = done or is_success also means termination is not synonymous with success. Because later rounds start wherever the counter happens to be, one wrong round relocates every subsequent index, so the reconstructed set is neither a subset nor a superset of the true one.

The conclusion is unchanged and now rests on the weaker claim that does hold: the visited set is outcome-dependent and not reliably reconstructible, so no offset can be shown to be held out. Evaluating tasks that were never trained on — what we actually ran — is immune either way. Found by adversarial review of our reply on huggingface/lerobot#4152.

• data/eval/eval_A_summary.json
• data/eval/eval_A_wave0_reanalysis.json
• scripts/repro_libero_init_state.py reproduces the init-state behaviour

Exploratory probe (unseen tasks, two pools) — hypothesis only

8 unseen tasks × 3 episodes = 24 episodes per arm per pool. grpo/ has the highest point estimate in both pools (29.2% in-dist, 20.8% held-out-camera, vs ≤12.5% for the others), but every interval includes zero, the winner is selected post hoc as the max over 4 arms × 2 pools with no multiplicity control, and 6 of GRPO's 7 in-dist successes come from just two tasks at 3/3 — excluding those two leaves 1/18, at or below SFT on the same six tasks. The two pools are different task sets, so their gap conflates viewpoint sensitivity with task difficulty. This is the hypothesis the preregistered test was written to decide, and it did not replicate. data/eval/eval_secondary_summary.json

How to load a checkpoint

Weights live in subfolders, so from_pretrained("<repo id>") will not work at the repository root. Download the arm you want, then load the local directory:

from huggingface_hub import snapshot_download
from lerobot.policies.smolvla.modeling_smolvla import SmolVLAPolicy

arm = "grpo"   # or "sft", "rs_sft", "dpo"
path = snapshot_download(
    "katsukiono/smolvla-libero-spatial-4arm",
    allow_patterns=f"{arm}/*",
)
policy = SmolVLAPolicy.from_pretrained(f"{path}/{arm}")

Pin the LeRobot commit in the repository's requirements.txt for exact reproduction. The preregistered run used lerobot 0.6.1 (1bb9933215dcb7ffeeae6d3746cda3f73f5a59e2), gymnasium 1.3.0. Evaluation wiring (environment construction, camera-name mapping, per-wave rebuild) is in src/eval_heldout.py; scripts/eval_unseen_prereg.sh runs the preregistered evaluation.

If you re-run the four-arm evaluation, note that scripts/eval_4arm.sh defaults its GRPO checkpoint to grpo_run7. The GRPO arm published here and used in every number above is run8. Override GRPO_CKPT accordingly.

Intended use and limitations

Research use only, in simulation.

• Trained and evaluated exclusively in LIBERO / robosuite. Never deployed on physical hardware; not validated for controlling a real robot.
• Proof-of-concept scale: n = 1 run per arm, one suite, 8 training tasks. No claim of generality across suites, embodiments, or seeds.
• grpo/ was trained with no KL anchor (kl_coef = 0); drift from the SFT policy increases monotonically over the run and performance plateaus late.
• grpo/'s reward requires privileged simulator state; the staged milestone detectors have task-dependent false negatives (~20% of successes carry no lift flag) and "grasped" means "grasp_attempted" (gripper-close command within 6 cm).
• camera_pose categories were deliberately excluded from SFT training, so held-out-viewpoint performance is expected to be degraded — but no evaluation in this study measures that gap cleanly (see the two-pool caveat above).
• GRPO here is short of updates, not data: 5× the rollouts of RS-SFT (1280 vs 256 episodes) for 20 optimizer steps against RS-SFT's 400. What the budget establishes is an exchange rate, not which learning rule is better.

License and redistribution basis

Released under Apache-2.0. What was actually checked, and the one gap:

• huggingface/lerobot — the training and inference code — is Apache-2.0 (verified via the GitHub license API).
• The SmolVLM2 backbone family (HuggingFaceTB/SmolVLM2-500M-Video-Instruct, …-256M-…) is tagged license:apache-2.0.
• Gap: the direct parent lerobot/smolvla_base carries no license field in its model-card frontmatter, no license: tag in its hub metadata, and no LICENSE file in the repo. This was verified, not assumed.
• What resolves it in practice: the LeRobot org publishes its own fine-tunes of that same base under Apache-2.0 — smolvla_libero_plus, smolvla_libero, smolvla_metaworld, smolvla_robotwin, smolvla_robomme, smolvla_robocerebra, smolvla_robocasa, smolvla_vlabench are all tagged license:apache-2.0 with base_model: lerobot/smolvla_base. The upstream rights-holder therefore treats derivative weights of smolvla_base as Apache-2.0 redistributable, and these checkpoints follow that precedent.
• SmolVLA pretraining used, per its authors, "compatibly licensed, open-source community-shared datasets under the lerobot tag".
• The fine-tuning code in the release repository is MIT.

If you are the upstream rights-holder and this reading is wrong, please open a discussion on this repository and the weights will be taken down.

Citation

Cite the preprint for the study and its findings, and the archived software record for the code and data:

@misc{ono2026smolvlarlpaper,
  author    = {Ono, Katsuki},
  title     = {Low-Budget Post-Training of a Flow-Matching VLA: Flow-SDE GRPO
               Diagnostics, an Evaluation Audit, and a Preregistered Null},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.21596932},
  url       = {https://doi.org/10.5281/zenodo.21596932},
  note      = {Preprint; not published or accepted anywhere}
}

@software{ono2026smolvlarl,
  author    = {Ono, Katsuki},
  title     = {SmolVLA\_RL: Low-Budget Flow-SDE GRPO Fine-Tuning of a
               Flow-Matching VLA},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.21585106},
  url       = {https://doi.org/10.5281/zenodo.21585106}
}

• Code, preregistration, raw per-episode data, analysis scripts: https://github.com/Ono-Katsuki/SmolVLA_RL
• Archived software record: https://doi.org/10.5281/zenodo.21585106
• Preprint: https://doi.org/10.5281/zenodo.21596932 — all versions; v1 is 10.5281/zenodo.21596933 (not published or accepted anywhere; no venue selected)

Upstream work these checkpoints build on: SmolVLA (arXiv:2506.01844), LIBERO-Plus (arXiv:2510.13626), π_RL / Flow-SDE (arXiv:2510.25889).