impact_libero_spatial

IMPACT — libero_spatial

Instruction-Modulated Perception + ACTion chunking, trained from scratch on the libero_spatial suite of LIBERO. No robot pretraining: an ImageNet ResNet-18 and a frozen T5-small are the only pretrained weights in the model.

libero_spatial is ten tasks that share one scene and differ only by a spatial relation ("the black bowl next to the ramekin"), so the instruction selects the target.

Result

Evaluated on 10 tasks × 20 initial states = 200 rollouts, at 256×256, control_mode=relative, executing the full 50-action chunk open-loop.

This is the 30,000-step checkpoint, selected as the highest-scoring of the four checkpoints evaluated for this suite. It is also the final checkpoint of its run, so no selection was applied.

How it compares

Averages over all four LIBERO suites. IMPACT's 78.6% is the mean of each suite's selected checkpoint; shipping every suite's final checkpoint instead gives 76.4%.

Baselines from Kim et al., Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success, Table I. IMPACT leads every policy that does not use robot pretraining, and both pretrained models of its own generation, at 78M parameters. The 90%+ tier is 3–7B models; the gap between OpenVLA at 76.5% and the same model at 90.2% comes from execution strategy, not capacity — which is the most promising thing to try here next, since IMPACT currently executes all 50 predicted actions open-loop.

Architecture

Two camera frames, an 8-D end-effector state and a natural-language instruction in; a 50-step action chunk out in one forward pass. The instruction enters twice — as 32 tokens appended to the transformer encoder sequence, and as FiLM scale/shift (1 + γ) ⊙ x + β on every ResNet stage, so language modulates perception and not only the fused memory.

Training

The suite split resolves each episode's instruction against the LIBERO benchmark's own per-suite task lists and raises on anything unplaceable, rather than splitting on task-index ranges.

Usage

from lerobot.policies.impact.modeling_impact import IMPACTPolicy

policy = IMPACTPolicy.from_pretrained("khanhnd61/impact_libero_spatial")

Requires the impact policy from LeRobot. Instructions are tokenized to a fixed 32 tokens; every LIBERO instruction fits (the longest, in libero_10, is 22 tokens), but longer instructions are silently truncated.

Limitations

• Single-suite. Trained on this suite's 10 tasks only; it is not a generalist policy.
• No held-out split. Training used every episode, so there is no validation signal separating learning from memorisation. The rollouts are the only out-of-sample evidence, and they use the same 10 tasks with different initial states.
• 200 rollouts per suite, not the 500 used by most published LIBERO tables, so sampling noise is roughly ±7 points (±2 SE).
• No FiLM ablation. Language use is measured, not assumed: on libero_goal, showing the policy another task's instruction drops it to 0.0% on the original goal over 600 episodes (control 88.5%) while it achieves the instructed goal 91.0% of the time. Which language path carries that is not isolated - shuffling perturbs FiLM and the encoder text tokens together - so use_film=false is still needed for the FiLM claim specifically.
• No test of language generalisation. Every instruction evaluated is one the policy trained on, so this is instruction following, not robustness to unseen wording.