ecot-openvla-7b-bridge-cd

ECoT OpenVLA 7B Bridge CD

This is an inference-only, fully fine-tuned Bridge checkpoint derived from Embodied-CoT/ecot-openvla-7b-bridge. It is packaged as a self-contained Transformers model; no adapter or separate base model is required.

Usage

Use the standard ECoT OpenVLA loader and replace only the model ID:

import torch
from transformers import AutoModelForVision2Seq, AutoProcessor

model_id = "Simsuk/ecot-openvla-7b-bridge-cd"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
vla = AutoModelForVision2Seq.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
).to("cuda")

instruction = "YOUR INSTRUCTION HERE"
prompt = (
    "A chat between a curious user and an artificial intelligence assistant. "
    "The assistant gives helpful, detailed, and polite answers to the user's questions. "
    f"USER: What action should the robot take to {instruction.lower()}? ASSISTANT: TASK:"
)
inputs = processor(prompt, image).to("cuda", dtype=torch.bfloat16)
action, generated_ids = vla.predict_action(
    **inputs,
    unnorm_key="bridge_orig",
    max_new_tokens=1024,
)

The repository executes custom model code when loaded with trust_remote_code=True; review the included Python files before use.

For a private or gated repository, authenticate with your own Hugging Face account before loading:

hf auth login

Official WidowX repository

The Transformers quickstart above is a model-ID replacement for the official ECoT quickstart. The included ecot_widowx_hf.patch adapts the official repository's legacy .pt loader to this Transformers model while retaining its existing WidowX environment and rollout loop. It targets official ECoT commit 1813ad76001f1e08095088f94a86c43fc0e457a3.

From the root of a clean official ECoT checkout:

pip install -U huggingface_hub
hf auth login
hf download Simsuk/ecot-openvla-7b-bridge-cd ecot_widowx_hf.patch \
  --local-dir /tmp/ecot-openvla-bridge-cd
git apply --check /tmp/ecot-openvla-bridge-cd/ecot_widowx_hf.patch
git apply /tmp/ecot-openvla-bridge-cd/ecot_widowx_hf.patch

python experiments/bridge/eval_model_in_bridge_env.py \
  --pretrained_checkpoint Simsuk/ecot-openvla-7b-bridge-cd \
  --host_ip ROBOT_INTERFACE_IP \
  --port ROBOT_INTERFACE_PORT

The patch adds a thin policy adapter that uses the official ECoT prompt, returns the seven-dimensional bridge_orig action, supplies decoded reasoning to the existing visualization, preserves the script's episode-reset interface, and makes the old gripper/bounding-box parser ignore primitives shortened to <|fim_pad|>. Review the patch before applying it, especially before controlling physical hardware.

Checkpoint

• Training run: bridge_tv_lam005_fimpad_fullftreasononlyskipon_cd6k_b128_daint2n_v1
• Step: 4,000
• Checkpoint type: full fine-tuning
• Skip token: <|fim_pad|> (token ID 32001)
• Action normalization key: bridge_orig
• Base revision: 492b3dbf3df380f6da333f86ce06dab028176166

This release was checked for complete source-matching weight shards, required custom-code and processor assets, atomic skip-token encoding, and isolated configuration/processor loading. It has not been gated on a new robot rollout evaluation.

License and attribution

The upstream ECoT code is MIT licensed. The pretrained model is derived from Llama 2 and therefore also inherits the Llama 2 Community License restrictions described by the upstream ECoT project. This repository retains the upstream model architecture and custom-code files. See the OpenVLA and Embodied Chain-of-Thought projects for their respective model, dataset, license, and usage documentation.