dish-utensil-fastwam-v2

Dish-utensil FastWAM v2

Public bundle for the dish-utensil manipulation model trained by George on the SleepMastger/dish-utensil-manipulation dataset (100 teleoperated Franka episodes: 50 fork + 50 spoon, 10 Hz, 42,863 frames).

This is the v2 relabel: the same 100 episodes, retrained with a new task string (below) that replaced the dataset's original instruction in meta/tasks.jsonl. Only the label and its cached T5 embedding differ from the v1 runs; frames, actions, and normalization are identical.

Status

This is the complete run: all 30 epochs (40,200 steps), checkpointed every 5 epochs. The model has not yet been evaluated on a robot.

Architecture: FastWAM full 30-layer video + 30-layer action MoT. Trained FROM SCRATCH (no LIBERO resume): Wan2.2 base video expert + ActionDiT linear-interp init, 30 epochs, global batch 32, cosine LR 1e-4, bf16 DeepSpeed ZeRO-1 on 4x GPU.

Checkpoints follow the same layout as the existing WAM repositories:

• checkpoints/weights/step_006700.pt — SHA-256 c69f0678f0f792a4e290bf7d0287ab4a7885c0f476d9264309fdf894395725c3
• checkpoints/weights/step_013400.pt — SHA-256 48f8563530caa824f5d07bdc9fa10ad12d60dc0518929b5a2fc2b468ea4f73cd
• checkpoints/weights/step_020100.pt — SHA-256 fb993fab227a6ecd92196e8322c386dd7461682bf209e6270855fb4f00bd9922
• checkpoints/weights/step_026800.pt — SHA-256 224d6779132c7b5ed3dcbaefcddb69e9cae90b45bd6b457ae3d604262b466f87
• checkpoints/weights/step_033500.pt — SHA-256 106d978710c03c97b924ff074ffa7551364897c83b7c3ee2deace66e84918526
• checkpoints/weights/step_040200.pt — SHA-256 3c56cd839cca91b761fddec0a3a7b81229864966a874e60df20acab13834eba8

Conditioning

Exact task text (byte-exact match required at deployment — the runtime looks the cached embedding up by SHA-256 of the formatted prompt):

pick the dishes out of the rack and place them on the table, then pick and place the spoon in the gray dish and fork in the pink dish

conditioning/text_embedding.pt is the exact cached T5 embedding consumed during training (context length 128, Wan2.2 text stack). The resolved training config sets load_text_encoder: false, so this cached tensor is part of the required inference bundle.

Input processing and normalization

• Two 256x256 RGB cameras (image = agentview, then wrist_image).
• Each camera is converted to a tensor and resized to 224x224.
• Cameras are concatenated horizontally to 224x448.
• Horizon: 33 observations; 32 action transitions at 10 Hz.
• State is 8-D: eef_xyz(3) + eef_rpy(3) + [gripper_width/2, -gripper_width/2] — REAL gripper widths (~0.0002..0.080 m), not binary; inference must feed real widths.
• Action is 7-D: delta XYZ, delta rotation XYZ, and gripper. In this dataset the rotation deltas are identically zero (translation-only teleop).
• Gripper action convention: raw robosuite {-1:open, +1:close} was remapped by the converter to {1:open, 0:close} (LIBERO/RLDS convention).
• Delta/padding mask is [true, true, true, true, true, true, false]; the gripper channel is absolute rather than delta.
• dataset_stats.json contains the exact min/max normalization statistics used by this run (recomputed on this dataset; no pretrained stats).

The exact preprocessing/model implementations are included under training_code/; the resolved config is config.yaml. No license is asserted here for the bundled upstream code; its original terms continue to apply.

Base components

This weights-only checkpoint is not standalone. It references Wan-AI/Wan2.2-TI2V-5B and requires the matching Wan VAE plus the included FastWAM code/configuration. PyTorch .pt files may contain pickled objects; load only in a trusted environment.