libero-metric-action-decoder-v2

LIBERO Metric Action Decoder v2

Legacy baseline: this standalone action-chunk transformer is preserved for reproducibility. The recommended successor is libero-metric-residual-servo-v5, which uses an analytic Cartesian controller plus a bounded learned residual and has narrow provider-free closed-loop controller evidence. The v2 and v5 weights are not architecture-compatible.

An experimental 3.18M-parameter transformer that maps Panda robot-state history and a structured metric subgoal to a 10-step chunk of normalized LIBERO actions.

Evidence boundary: this checkpoint has held-out open-loop action prediction results only. It has not demonstrated improved closed-loop task success, real-robot performance, safety, or cross-embodiment transfer.

What this model is

The decoder is the low-level half of a planner/controller split:

external planner or VLM -> 28D metric subgoal -> this decoder -> LIBERO actions
                               + 4 robot states

It takes four recent 8D proprioceptive states plus one 28D metric waypoint and predicts ten 7D actions in one forward pass. It can sit behind a VLM that produces the documented metric-subgoal contract, but this checkpoint itself contains no VLM and receives no pixels, language, or VLM hidden states.

This model is therefore best described as an embodiment-specific metric action decoder, not a standalone VLA and not a general-purpose action expert.

Architecture

Four projected state tokens and one projected subgoal token are concatenated with ten learned action queries. A transformer encoder processes all tokens, and separate heads produce six motion values plus one gripper logit for every action query.

Input and output contract

Robot state: [batch, 4, 8]

Each state uses this order:

1. end-effector position (x, y, z) in metres, in the LIBERO robot-base frame;
2. end-effector axis-angle orientation (rx, ry, rz) in radians;
3. two Panda gripper joint positions.

The four states must be chronological. Training statistics are embedded in the Safetensors checkpoint and duplicated in normalization.json for inspection.

Metric subgoal: [batch, 28]

The feature order is:

1. eight-way phase one-hot: reach, align, grasp, manipulate, release, retract, hold, done;
2. absolute target position (x, y, z) in robot-base metres;
3. unit approach direction (x, y, z);
4. unit gripper-opening direction (x, y, z);
5. three-way gripper target one-hot: open, closed, hold;
6. position tolerance in metres;
7. orientation tolerance in radians;
8. normalized speed;
9. expected-contact flag;
10. confidence;
11. packet age divided by five seconds and clipped to [0, 1];
12. remaining validity divided by five seconds and clipped to [0, 1];
13. validity flag.

metric_subgoal.py implements the exact vectorization order, and metric_subgoal.schema.json defines the semantic JSON packet.

Action chunk: [batch, 10, 7]

The first six values are normalized LIBERO operational-space motion commands. The seventh is the gripper command: +1 means close and -1 means open. decode_actions clips motion to [-1, 1] and binarizes the gripper output.

These are not joint torques, joint positions, or universal robot commands.

Quick start

git clone https://huggingface.co/hydrogenbondd007/libero-metric-action-decoder-v2
cd libero-metric-action-decoder-v2
python -m pip install -r requirements.txt
python example_inference.py --model .

Minimal Python usage:

import torch

from metric_subgoal import vectorize_metric_subgoal
from model import load_model

model = load_model(".")
state_history = torch.zeros(1, 4, 8)  # replace with real chronological state
packet = {
    "schema_version": "frontier_metric_subgoal_v1",
    "observation_id": "example-observation-1",
    "measured_at_ms": 1000,
    "valid_for_ms": 1000,
    "phase": "reach",
    "position_robot_m": {"x": -0.10, "y": 0.05, "z": 1.02},
    "approach_robot_unit": {"x": 0.0, "y": 0.0, "z": -1.0},
    "opening_robot_unit": {"x": 0.0, "y": 1.0, "z": 0.0},
    "gripper_target": "open",
    "position_tolerance_m": 0.01,
    "orientation_tolerance_rad": 0.10,
    "speed": 0.25,
    "expected_contact": False,
    "confidence": 0.9,
    "max_control_steps": 10,
}
subgoal = vectorize_metric_subgoal(packet, available_at_ms=1000).unsqueeze(0)

with torch.inference_mode():
    action_chunk = model.decode_actions(state_history, subgoal)

The zero state in this short snippet demonstrates the API shape only; use the complete example or real LIBERO observations for meaningful outputs.

Training data

The checkpoint was trained from physical-intelligence/libero revision 20f6ab338acbb455b4215db2ed5f7f1ad855f036.

The selected tasks were:

• put the bowl on the plate;
• turn on the stove;
• open the middle drawer of the cabinet.

Training subgoals were constructed with hindsight from future expert states during preprocessing. Expert future information is not an input to the model at inference time; a runtime planner must supply the metric subgoal. Pixels were not retained or used for training.

See dataset_manifest.json, training_config.json, and provenance.json for the pinned split, label conventions, hyperparameters, source hashes, and environment.

Evaluation

Evaluation used held-out episodes from the same three tasks.

Zeroing the subgoal increased motion MAE by 0.09056; shuffling it increased motion MAE by 0.12600. These controls show that the network uses the metric subgoal. They do not establish planning quality or robotic task success.

The full machine-readable result is in evaluation_results.json. That file marks the artifact benchmark_ready: false because closed-loop task-success evaluation has not yet passed.

Intended use

• research on planner/controller interfaces;
• LIBERO simulator experiments with Panda-compatible state and action schemas;
• open-loop action prediction baselines;
• studies of small decoders behind external VLM planners.

Limitations and out-of-scope use

• No closed-loop task-success claim.
• No real-robot validation.
• No collision-avoidance or force-safety guarantee.
• No visual or language understanding inside this checkpoint.
• Only three LIBERO Goal tasks were used for training and validation.
• The weights are tied to Panda/LIBERO coordinates, normalization, action semantics, and a 10 Hz data clock.
• Cross-robot transfer requires a new embodiment adapter or retraining.
• The ten predicted steps have not been validated for blind open-loop execution; receding-horizon use still requires closed-loop evaluation.

Do not deploy this model directly on physical hardware. A hardware integration must independently validate coordinate frames, units, control rate, workspace limits, collision handling, command clipping, and emergency-stop behavior.

Reproducibility and file integrity

The release uses model.safetensors; pickle checkpoints are intentionally not included. The exported weights contain 70 tensors and were checked for an exact round trip from the selected checkpoint.

• model.safetensors SHA-256: 497db1d591707c1a471d79d27887366094a10ce89eee5f8622fcf445b5063adb
• source training checkpoint SHA-256: 1f3dc03e9b88380aff149899d174b1b36654fb11d6ca30cb4225ff4e3547fbfb

See MANIFEST.sha256 for release-file hashes.

License and attribution

The model code and weights are released under Apache 2.0. The training dataset is separately published under CC BY 4.0. See NOTICE.md for attribution and cite the LIBERO paper in published work.

Version

This is v0.1.0-experimental, published on 2026-08-31.