pusht-fastwam

Push-T FastWAM (from scratch, 30 epochs)

Public archival bundle for a FROM-SCRATCH run on the real-hardware Push-T task: push an orange T-shaped block across a table until it aligns with a pink T outline, on a Franka Emika Panda. The gripper is clamped shut on a marker pen that acts as a single-point pusher, so this is the classic Push-T task run on real hardware rather than in simulation.

Trained on [SleepMastger/pusht-manipulation] (https://huggingface.co/datasets/SleepMastger/pusht-manipulation) — 100 human teleoperation demonstrations, 32,131 frames at 10 Hz, converted to LeRobot v2.1. Companion model: SleepMastger/pusht-flashwam — identical data and recipe, different architecture, so the two are directly comparable.

Architecture: Original FastWAM, full non-decoupled MoT (~6.0B): 30-layer Wan2.2 video expert (hidden 3072) + 30-layer action expert (hidden 1024), no KV fusion. Action expert initialised from ActionDiT_linear_interp_Wan22_alphascale_1024hdim.pt.

Training

Completed all 30 epochs. Checkpoints were saved every 5 epochs:

• checkpoints/weights/step_005025.pt (epoch 5) — SHA-256 c230b31fa8ccdc56beea7e6aab6e8acef74306c541b5ab4b554000c9f91d6d1d
• checkpoints/weights/step_010050.pt (epoch 10) — SHA-256 da9fb58a5fb2c83ac4680154c9c41787c75a9695939d9ed1491d43a377e39636
• checkpoints/weights/step_015075.pt (epoch 15) — SHA-256 2e11bbb446bfd304e1879d3e2e8d86375adeb0a9bfde3cc81a1a145a476f2995
• checkpoints/weights/step_020100.pt (epoch 20) — SHA-256 3aebaf3f174428838000d465cc2c79c9f8e158860ede2ec69e15ba22d6aa335d
• checkpoints/weights/step_025125.pt (epoch 25) — SHA-256 67af75cd386dad0efb7c2dcc76f7226017df38f4fd6a573b21e54ebaa9722176
• checkpoints/weights/step_030150.pt (epoch 30) — SHA-256 ac556b5be985384a03e575bec8cefa5f284e6160f725722dadbfed373c798125

The model has not been evaluated on a robot.

Conditioning

Exact task text (must byte-match at deployment):

push the T block to the target outline

No trailing period. conditioning/text_embedding.pt is the exact cached T5 embedding consumed during training (Wan text stack, context length 128); its filename hash in the training cache is 71cb088954da46da4e3cb6c6f73ac812690e9bccf886c7858377c61918b4e3e8. The resolved config sets load_text_encoder: false, so this cached tensor is part of the required inference bundle.

Input processing and normalization

• Two 256x256 RGB cameras with raw LeRobot keys image / wrist_image, resized to 224x224 and concatenated horizontally to 224x448.
• Dataset is LeRobot v2.1 (per-episode parquet + AV1 mp4), staged node-locally by training_code/stage_pusht_local.sh.
• Horizon: 33 observations; 32 action transitions at 10 Hz.
• State is 8-D: eef_pos(3) + quat2axisangle(eef_quat)(3) + [gripper_width/2, -gripper_width/2].
• Action is 7-D: delta XYZ (metres per 100 ms step), delta rotation XYZ, grasp. Raw grasp uses the robosuite {-1 open, +1 close} convention and is remapped g -> (1-g)/2 to the LIBERO/RLDS {1 open, 0 close} convention at conversion.
• dataset_stats.json holds the exact min/max statistics used by this run.

⚠️ Six of the fifteen channels are constant — read before deploying

The teleoperation rig commanded translation only, and the pen stayed clamped for the whole task. Measured across all 100 episodes / 32,131 frames:

This does not produce NaNs. SingleFieldLinearNormalizer (training_code/normalizer.py) has ignore_dim = input_range < range_tol with range_tol = 1e-4, which catches all six — the tightest, the gripper state dims at 3.2e-05, clears it by about 3x. An ignored dim gets scale = 1.0 and offset = -min, i.e. it normalizes to x - min: exactly 0 for the four action dims whose raw range is exactly 0.0, and [0, 3.2e-05] for the two gripper state dims. Finite and negligible beside the ±1 live channels either way.

Consequences for deployment:

1. Only dx, dy, dz are meaningful outputs. Dims 3-6 were constant in training, so the model has learned to emit their constant value; denormalize as usual and they come back as the constants. Do not interpret them as commanded rotation or grasp.
2. The gripper must stay closed. The policy never saw it open and cannot command it to. Clamp the pusher before handing control to the policy.
3. Do not lower range_tol below 3.2e-05 — the gripper state dims would then be treated as live and divide by ~0.

The full 7-D action / 8-D proprio space was kept deliberately, so the architecture is byte-identical to the dish-utensil and fruit-picking runs and the results stay comparable.

Attention masks

• video_attention_mask_mode: first_frame_causal
• action_group_causal_mask_mode: group_diagonal
• Text cross-attention is enabled for the action expert.

The exact implementations, configs, dataset converter, staging and launch scripts are included under training_code/; the resolved config is config.yaml. training_code/preflight.py is the CPU-only check that verifies the tensor contract and the degenerate-channel behaviour described above. No license is asserted here for the bundled upstream code; its original terms continue to apply.

Base components

This weights-only checkpoint is not standalone. It references Wan-AI/Wan2.2-TI2V-5B and requires the matching Wan VAE plus the included FastWAM code/configuration. PyTorch .pt files may contain pickled objects; load only in a trusted environment.