Downloads · 30 days
0
OS_libero_traj_strict_rew
vla-manan/OS_libero_traj_strict_rew
OS_libero_traj_strict_rew is a machine learning model from vla-manan. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
OpenSora + OpenVLA-OFT checkpoints trained with GRPO on LIBERO-Spatial using a strict VLM trajectory reward (Jadversarialfast prompt).
Access
Public
Updated Mar 9, 2026
Repo size
121 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.distcp90.6 GB · 75%
At a glance
- Access
- Public
- Created
- Mar 9, 2026
- Updated
- Mar 9, 2026
- SHA
- 6ee975ef
Model card
From the Hugging Face model README
OS_libero_traj_strict_rew
OpenSora + OpenVLA-OFT checkpoints trained with GRPO on LIBERO-Spatial using a strict VLM trajectory reward (J_adversarial_fast prompt).
Training Details
- Base model: RLinf-OpenVLAOFT-LIBERO-130-Base-Lora (OpenVLA-OFT with LoRA)
- RL algorithm: GRPO (Group Relative Policy Optimization)
- Environment: LIBERO-Spatial (10 tasks)
- Reward model: Qwen3-VL-8B-Instruct with J_adversarial_fast prompt
- Reward type: VLM trajectory reward (4 frames sampled from trajectory)
- Training framework: RLinf
- GPUs: 8x NVIDIA B200
- KL penalty (kl_beta): 0.0
- Reward coefficient: 5.0
- VLM max_new_tokens: 32
- enable_thinking: False
- val_check_interval: 5
Checkpoints
| Checkpoint | Description |
|---|---|
global_step_25/ | Step 25 — peak eval performance window |
global_step_30/ | Step 30 — peak eval performance window |
Each checkpoint contains:
actor/model_state_dict/full_weights.pt— consolidated full model weights (~15GB)actor/dcp_checkpoint/— distributed checkpoint shards for resuming training (~43GB)
VLM Reward Prompt
See prompt.txt for the full J_adversarial_fast prompt template used during training.
Prompt Metrics
| VLM Model | Accuracy | Precision | Recall | F1 | FP Rate |
|---|---|---|---|---|---|
| Qwen3-VL-8B | 68.3% | 90.6% | 59.0% | 0.714 | 12.8% |
| Qwen3.5-9B | 85.8% | 97.1% | 81.5% | 0.886 | 5.1% |
Training Eval Curve (VLM success)
Training ran for 95 steps with periodic validation every 5 steps.
- Peak VLM success ~85% around steps 20-25
- Gradual decline after step 30
Usage
To load the full weights for inference:
import torch
state_dict = torch.load("global_step_25/actor/model_state_dict/full_weights.pt", map_location="cpu")
# Apply to your OpenVLA-OFT model
License
See the main RLinf repository for license details.
- Created
- Mar 9, 2026
- Updated
- Mar 9, 2026