dit-diffusion-dinov3-xarm-blue-mug-v0

Multi-Task DiT (diffusion) + DINOv3 ViT-B/16

LeRobot Multi-Task DiT policy checkpoints with a DINOv3 ViT-B/16 vision encoder (text encoder stays CLIP), uploaded by goal_gen/upload_hf_checkpoints_dinov3.sh. Each checkpoint-*/ subfolder is the deployment-ready pretrained_model/ payload (model.safetensors + config.json + pre/postprocessor + train_config.json).

⚠️ Loading requires the bundled patch

These checkpoints set vision_encoder_name="facebook/dinov3-vitb16-pretrain-lvd1689m" in config.json, which stock lerobot rejects. This repo ships the runtime monkey-patch multitask_dit_dinov3_patch/ at its root; import it once before from_pretrained and loading works with no gated HuggingFace access (the DINOv3 architecture is built locally and the fine-tuned weights come from the checkpoint).

import sys
from huggingface_hub import snapshot_download

# Download the patch package + one checkpoint.
local = snapshot_download(
    "JayCao99/dit-diffusion-dinov3-xarm-blue-mug-v0",
    allow_patterns=["multitask_dit_dinov3_patch/*", "checkpoint-100000/*"],
)

sys.path.insert(0, local)          # make the patch package importable
import multitask_dit_dinov3_patch            # noqa: F401 — applies the patch

from lerobot.policies.multi_task_dit.modeling_multi_task_dit import MultiTaskDiTPolicy
policy = MultiTaskDiTPolicy.from_pretrained(f"{local}/checkpoint-100000")
policy.eval()

For lerobot CLI tools (lerobot-train, lerobot-eval, ...), instead pass --policy.discover_packages_path=multitask_dit_dinov3_patch (with the package on PYTHONPATH).

See multitask_dit_dinov3_patch/README.md in this repo for details.