impact_libero_goal

IMPACT — libero_goal

Instruction-Modulated Perception + ACTion chunking, trained from scratch on the libero_goal suite of LIBERO. No robot pretraining: an ImageNet ResNet-18 and a frozen T5-small are the only pretrained weights in the model.

libero_goal is ten different goals in a single shared kitchen scene, distinguishable only by the instruction.

Result

Evaluated on 10 tasks × 20 initial states = 200 rollouts, at 256×256, control_mode=relative, executing the full 50-action chunk open-loop.

This is the 25,000-step checkpoint, selected as the highest-scoring of the four checkpoints evaluated for this suite. The final checkpoint of the same run scores 84.5%; this one was picked because it scored higher on the reported rollouts, which means the 89.0% figure carries selection bias and 84.5% is the unselected number.

How it compares

Averages over all four LIBERO suites. IMPACT's 78.6% is the mean of each suite's selected checkpoint; shipping every suite's final checkpoint instead gives 76.4%.

Baselines from Kim et al., Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success, Table I. IMPACT leads every policy that does not use robot pretraining, and both pretrained models of its own generation, at 78M parameters. The 90%+ tier is 3–7B models; the gap between OpenVLA at 76.5% and the same model at 90.2% comes from execution strategy, not capacity — which is the most promising thing to try here next, since IMPACT currently executes all 50 predicted actions open-loop.

Architecture

Two camera frames, an 8-D end-effector state and a natural-language instruction in; a 50-step action chunk out in one forward pass. The instruction enters twice — as 32 tokens appended to the transformer encoder sequence, and as FiLM scale/shift (1 + γ) ⊙ x + β on every ResNet stage, so language modulates perception and not only the fused memory.

Training

The suite split resolves each episode's instruction against the LIBERO benchmark's own per-suite task lists and raises on anything unplaceable, rather than splitting on task-index ranges.

Usage

from lerobot.policies.impact.modeling_impact import IMPACTPolicy

policy = IMPACTPolicy.from_pretrained("khanhnd61/impact_libero_goal")

Requires the impact policy from LeRobot. Instructions are tokenized to a fixed 32 tokens; every LIBERO instruction fits (the longest, in libero_10, is 22 tokens), but longer instructions are silently truncated.

Limitations

• Single-suite. Trained on this suite's 10 tasks only; it is not a generalist policy.
• No held-out split. Training used every episode, so there is no validation signal separating learning from memorisation. The rollouts are the only out-of-sample evidence, and they use the same 10 tasks with different initial states.
• 200 rollouts per suite, not the 500 used by most published LIBERO tables, so sampling noise is roughly ±7 points (±2 SE).
• No FiLM ablation. Language use is measured, not assumed: on libero_goal, showing the policy another task's instruction drops it to 0.0% on the original goal over 600 episodes (control 88.5%) while it achieves the instructed goal 91.0% of the time. Which language path carries that is not isolated - shuffling perturbs FiLM and the encoder text tokens together - so use_film=false is still needed for the FiLM claim specifically.
• No test of language generalisation. Every instruction evaluated is one the policy trained on, so this is instruction following, not robustness to unseen wording.