egopi-ab2x2-w0p03

EGoPi 2×2 Factorial — Cognitive Alignment × Action-Latent Tokenizer

Four checkpoints from a controlled 2×2 factorial study. All four arms share identical code, data, seed, batch size, and step count. Exactly two binary switches differ between them.

Axes

Cognitive alignment (A axis). Grounds the vision-language cognitive tokens in the robot's physical space through a 256-hidden-unit adapter with a linear latent-alignment head and a 24-dimensional fingertip target (weight 0.2).

Action-latent tokenizer (B axis). Uses an action tokenizer trained with VICReg regression weight 0.03 (embodreg_vicreg_w0p03), which reduces the human↔robot action-latent gap. When this switch is on, the flow-matching target is the aligned latent z rather than the native {p, r, q} representation.

Training

• Backbone: VTC-Qwen3-VL (4.679B) + lm_head (0.630B) + action model / MSAT (1.245B trainable)
• Total 6.913B parameters, BF16, DeepSpeed ZeRO-2
• 30,000 steps, effective batch size 64 (2 GPUs × 16 per device × 2 gradient accumulation)
• Data seed 0
• Human tasks (5): bottle, box, ball, doll, towel
• Robot tasks (4, OpenArm v3): bottle, cup, doll, snack

Results

Parameter counts are exactly additive across the two axes (722 + 7 + 2 = 731), confirming the two switches are structurally orthogonal.

Reading the loss values

Loss is only comparable within a tokenizer setting. The B axis replaces the flow-matching target, so N/A and B/AB optimize different quantities. Comparing 0.0248 against 0.4557 says nothing about relative model quality.

Valid within-axis comparisons:

• N → A: 0.0379 → 0.0248 (cognitive alignment, −35%)
• B → AB: 0.5379 → 0.4557 (cognitive alignment, −15%)

Cognitive alignment lowers training loss under both tokenizer settings, which is a consistent signal — but training loss alone does not establish improved policy performance. Cross-axis comparison requires downstream evaluation.

Contents

Each arm folder holds final weights only — no optimizer state, no RNG state, no intermediate checkpoints.

<arm>/
├── config.json
├── model-0000{1,2,3}-of-00003.safetensors
├── model.safetensors.index.json
├── training_args.bin
├── processor/
└── experiment_cfg/