openvla-7b-libero-spatial-lora

OpenVLA-7B LoRA · LIBERO-Spatial (2000 steps, study project)

LoRA fine-tune of openvla/openvla-7b on the LIBERO-Spatial suite (libero_spatial_no_noops from openvla/modified_libero_rlds).

⚠️ This is a learning-project checkpoint, not a strong model. It was trained for only 2000 steps under a deliberately cost-constrained setup, and evaluates far below the official openvla/openvla-7b-finetuned-libero-spatial (~84.7%). It is published for reproducibility of the process, not for use. Numbers and the reasons behind them are stated honestly below.

Setup

Revision step1000 holds the intermediate 1000-step (with-aug) checkpoint.

Evaluation

Official harness experiments/robot/libero/run_libero_eval.py, LIBERO-Spatial, seed 7, MUJOCO_GL=egl, --center_crop False (matching the no-aug final training stage).

All successes came from a single task (pick up the black bowl on the cookie box, 2/3); the other seven tasks scored 0/3. LIBERO-Spatial keeps the object fixed and varies its placement, so this looks like 2000 steps memorizing one spatial configuration rather than learning a general locate → grasp → place policy.

Why it underperforms (honest post-mortem)

1. Only 2000 steps. Training was stopped at a loss plateau to stay inside a small budget; the official checkpoint is trained far longer.
2. Mixed preprocessing. First 1000 steps used image augmentation, the last 1000 did not, so the model saw inconsistent inputs and neither center_crop setting at eval matches cleanly.
3. Merge-and-restart lost optimizer state. To work around a CPU-side data-pipeline bottleneck, the step-1000 LoRA was merged and a fresh LoRA started, discarding Adam momentum; with a constant lr of 5e-4 the loss then oscillated on a plateau (~2.4–3.0) rather than continuing to descend.

What to do differently: keep image_aug=True throughout (parallelize the data pipeline or use multiple GPUs to absorb the cost), run one continuous training job without merge-restarts, and train until the loss genuinely converges rather than stopping at the first plateau.

Usage

from transformers import AutoModelForVision2Seq, AutoProcessor

processor = AutoProcessor.from_pretrained("duanxingjuan/openvla-7b-libero-spatial-lora", trust_remote_code=True)
vla = AutoModelForVision2Seq.from_pretrained(
    "duanxingjuan/openvla-7b-libero-spatial-lora",
    torch_dtype="bfloat16", trust_remote_code=True,
).to("cuda")

action = vla.predict_action(**inputs, unnorm_key="libero_spatial_no_noops", do_sample=False)

Not an official OpenVLA release. Part of a personal OpenVLA study repo.