openvla-libero-spatial-checkpoints

OpenVLA LIBERO-Spatial Training Checkpoints

这是OpenVLA在LIBERO-Spatial数据集上训练的完整checkpoint集合，包含训练过程中的多个epoch checkpoints。

📦 包含的Checkpoints

本仓库包含 10 个训练checkpoints：

🚀 使用方法

加载特定epoch的checkpoint

from transformers import AutoModelForVision2Seq, AutoProcessor
import torch

# 选择要使用的checkpoint（例如epoch 40）
checkpoint_path = "{{repo_id}}/epoch-40-step-005480"

# 加载模型
model = AutoModelForVision2Seq.from_pretrained(
    checkpoint_path,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16
).to("cuda")

# 加载processor
processor = AutoProcessor.from_pretrained(
    checkpoint_path,
    trust_remote_code=True
)

# 预测动作
from PIL import Image

image = Image.open("observation.jpg")
prompt = "In: What action should the robot take to pick up the object?\nOut:"
inputs = processor(prompt, image).to("cuda", dtype=torch.bfloat16)

action = model.predict_action(
    **inputs, 
    unnorm_key="libero_spatial_no_noops", 
    do_sample=False
)
print(action)  # 7-DoF action vector

在评估脚本中使用

# 使用epoch 40的checkpoint
python experiments/robot/libero/run_libero_eval.py \
    --model_family openvla \
    --pretrained_checkpoint {{repo_id}}/epoch-40-step-005480 \
    --task_suite_name libero_spatial_no_noops \
    --center_crop False \
    --num_trials_per_task 50

下载到本地使用

# 下载特定checkpoint
huggingface-cli download {{repo_id}} --include "epoch-40-step-005480/*" --local-dir ./local-checkpoints

# 或下载所有checkpoints
huggingface-cli download {{repo_id}} --local-dir ./local-checkpoints

📊 训练信息

• Base Model: OpenVLA (Prismatic + DinoSigLIP-224px)
• Training Dataset: LIBERO-Spatial (no noops)
• Framework: Transformers + PyTorch
• Training Hardware: 8x GPU
• Batch Size: 384 (global)

🎯 推荐使用

• 最佳性能: 使用最后几个epoch的checkpoints（epoch 35-40）
• 快速验证: 使用早期checkpoints（epoch 5-10）
• 对比实验: 比较不同epoch的性能变化

📈 如何选择Checkpoint

1. Epoch 40: 训练最充分，推荐用于最终评估
2. Epoch 25-36: 性能稳定，适合大多数任务
3. Epoch 10-20: 早期checkpoint，适合快速实验
4. Epoch 5: 初始训练，主要用于对比

📝 文件结构

libero-spatial/
├── epoch-05-step-000685/
│   ├── config.json
│   ├── model-*.safetensors
│   ├── preprocessor_config.json
│   ├── tokenizer_config.json
│   └── dataset_statistics.json
├── epoch-10-step-001370/
│   └── ...
├── epoch-15-step-002055/
│   └── ...
... (其他checkpoints)

🔧 环境要求

pip install transformers torch pillow huggingface_hub

📖 引用

@article{kim2024openvla,
  title={{OpenVLA: An Open-Source Vision-Language-Action Model}},
  author={{Kim, Moo Jin and Pertsch, Karl and Karamcheti, Siddharth and Xiao, Ted and Balakrishna, Ashwin and Nair, Suraj and Rafailov, Rafael and Foster, Ethan and Lam, Grace and Sanketi, Pannag and Nasiriany, Soroush and Liang, Zheyuan and Sadigh, Dorsa and Levine, Sergey and Liang, Percy}},
  journal={{arXiv preprint arXiv:2406.09246}},
  year={{2024}}
}

📄 License

MIT License

🔗 相关链接

• OpenVLA项目: https://github.com/openvla/openvla
• LIBERO项目: https://github.com/Lifelong-Robot-Learning/LIBERO
• Paper: https://arxiv.org/abs/2406.09246