apex-0-7-liquid-effector-set-450m

Apex 0.7 Liquid Effector-Set Tri-Attention (450M)

Apex 0.7 generalizes Apex 0.6 from a fixed two-effector (left/right hand) setup into a variable effector-set contract. The model is a short-horizon bimanual robot motion policy built from a LiquidAI LFM2-VL-450M vision-language backbone, a LoRA adapter, and an Apex effector interaction + Geodesic Flow Matching (GFM) policy head.

This repository contains a research checkpoint from a 2000-step training run (not a complete robot controller). Frame conversion, IK, collision checks, speed limits, and hardware execution must remain outside the model.

Base Model

LiquidAI/LFM2-VL-450M

Files

apex_0_7_effector_policy.pt
	Apex 0.7 effector-interaction policy head weights + (legacy) motion decoding weights.

liquid_lora_adapter/
	PEFT LoRA adapter for LiquidAI/LFM2-VL-450M.

apex_0_7_effector_manifest.json
	Training configuration and final metrics.

metrics.jsonl
	Training metrics log (logged during the 2000-step run).

Architecture (high level)

The runtime uses Liquid’s VLM world K/V hooks together with effector tokens.

In Apex 0.7:

• Effector interaction transforms an effector_set into effector tokens (self-attention among effectors + attention to the world/motion streams).
• Effector tokens are projected into attention K/V and appended into the motion GFM attention pipeline.
• The GFM motion head performs attention over the coupled K/V stream (i.e., a real cross-attention path from VLM world K/V into the GFM blocks), then integrates the predicted flow over a short horizon.

If you want to verify the coupling in code, see:

• models/apex_1/apex1/effector_policy.py
• models/apex_1/apex1/cross_attention_flow.py

Training Data

Training used the local Apex effector-set mix built from ABC130k prefetch:

local_datasets/apex_mixes/apex07_effector_from_abc130k_prefetch_20260705T0505Z/delta_windows.jsonl

Dataset composition at indexing time (manifest):

records_indexed: 429,105
train_records: 420,523
min_records_gate: 100,000
record_schema: apex.delta_window.v3
max_effectors: 2

Effector labels (proxy for current dataset content):

unknown, open, pre_contact, closing, stable_grasp, micro_adjust, release

Training Configuration

Manifest key hyperparameters:

steps: 2000
model_dim: 512
layers: 6
heads: 2
head_dim: 96
kv_heads: 8
hook_layer_indices: 1,3,5,7,9,11
prismatic: false
effector_lambda: 0.15
action_spec: apex_action_20d_bimanual_tcp_scalar_gripper_v1

Metrics

Final training step (manifest):

final step: 2000
loss: 0.1341955065727234
loss_flow_mse: 0.05861161055509001
loss_effector_total: 0.5038926561828703
loss_effector_phase: 0.46644264459609985
loss_effector_contact: 0.14249000925337896
loss_effector_grasp: 0.002598094055429101
loss_effector_release: 0.004711934045189992
elapsed_seconds: 5236.949531555176

Intended Use

Use this checkpoint as a motion proposal model for short-horizon robot trajectory/action generation.

Inputs in this project follow the Apex effector-set contract (schema):

apex.effector_set_contract.v1
record_schema: apex.delta_window.v3

The model still decodes into a legacy bimanual 20D action spec in this particular checkpoint.

This repo does not include a merged/ready-to-run robot controller. Wrap it with:

• safety, workspace limits
• collision checks
• IK/frame conversion
• speed/acceleration constraints

Limitations

• This is a research checkpoint; it is not a complete autonomy stack.
• Effector supervision is proxy/heuristic based on current dataset labels.
• In this checkpoint, motion is still decoded through the legacy 20D bimanual action representation until a native N-effector action spec exists.

Safety

Use only as a motion proposal model. Enforce workspace limits, collision checking, speed/acceleration constraints, IK validity, and emergency stop behavior in downstream robot software.