hunyuan15-robotwin-r64attn-r16ffn

HunyuanVideo-1.5 RoboTwin LoRA — attention r64 / video-FFN r16

MeanFlow LoRA post-training of the step-distilled HunyuanVideo-1.5 480p I2V model on the RoboTwin bimanual (ALOHA-AgileX) manipulation dataset.

This variant inverts the usual rank allocation: attention (Q/K/V/O) gets rank 64 while the video-stream FFN gets rank 16. The sibling recipe it was derived from uses 16/32. The motivation is that subtask prompts for most of the 15 RoboTwin tasks differ only by a left/right-arm minimal pair (e.g. handover_block's "…with the left arm" vs "…with the right arm" over the same scene), so the hypothesis was that the token-to-spatial-region correspondence is attention-limited rather than FFN-capacity-limited.

Contents

Configuration

Data

RoboTwin 2, randomized_500 subset, 15 tasks, ALOHA-AgileX embodiment, head camera. Dense sliding windows at stride 16, target size 432, spatial prompt gating applied to 75% of eligible samples. Per-task prompt templates with dynamic subtask conditioning.

Training

Trained on 4× NVIDIA H200 NVL over two legs (a 24 h wall-clock limit split the run; the second resumed from checkpoint-011057):

Training future_loss fell from ~1.50 to ~0.6 and was stable there from roughly step 6,500 onward; the final logged value was 0.655.

Caveats

• Quality has not been quantitatively evaluated. Fifteen per-task validation videos were generated from the final checkpoint and inspected only as a loss-curve sanity check. There are no success-rate, Physics-IQ, or human-preference numbers behind this checkpoint, and no A/B against the 16:32 sibling. Treat the rank-allocation hypothesis above as untested.
• The adapter is trained against the step-distilled HunyuanVideo-1.5 variant and its 4-step MeanFlow sampler. It is not expected to transfer to the non-distilled base.
• Use of this adapter is subject to the licence terms of the base HunyuanVideo-1.5 model, which are not superseded here.

Usage

from diffusers import HunyuanVideo15ImageToVideoPipeline

pipe = HunyuanVideo15ImageToVideoPipeline.from_pretrained(
    "hunyuanvideo-community/HunyuanVideo-1.5-Diffusers-480p_i2v_step_distilled",
    torch_dtype="bfloat16",
)
pipe.load_lora_weights(
    "dave7k/hunyuan15-robotwin-r64attn-r16ffn",
    weight_name="lora/pytorch_lora_weights.safetensors",
)