Downloads · 30 days
21
22% of all-time downloads
mu-vla/mu-vla-openvla-oft-mikasa-robo-5-tasks-no-memory
mu-vla-openvla-oft-mikasa-robo-5-tasks-no-memory is a robotics model from mu-vla. Use it for the robotics task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
This is the memoryless control for the mu-VLA checkpoints: plain OpenVLA-OFT, with no memory tokens and no recurrent state, fine-tuned on the same five MIKASA-Robo-VLA environments, with the same episodic dataloader,…
Downloads · 30 days
21
22% of all-time downloads
All-time downloads
97
Public
Parameters
7.5B
15.9 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors15.6 GB · 98%
From the Hugging Face model README
This is the memoryless control for the mu-VLA checkpoints: plain OpenVLA-OFT, with no memory tokens and no recurrent state, fine-tuned on the same five MIKASA-Robo-VLA environments, with the same episodic dataloader, for the same number of steps. It exists so that any gain reported by a mu-VLA checkpoint can be attributed to the memory module rather than to the data, the task mixture or the fine-tuning recipe.
The model is a per-step policy: every action is predicted from the current observation alone. On the memory-heavy MIKASA-Robo tasks, where the cue is shown early in the episode and is no longer visible when the action has to be taken, this is the behaviour a memory mechanism has to beat.
This checkpoint corresponds to experiment 2 in the mu-VLA experiment matrix, at training step 150000.
Sibling checkpoints, same five environments, with recurrent memory:
Base model: openvla/openvla-7b, fine-tuned with the OpenVLA-OFT recipe (L1 regression action
head, proprioception projector, two input camera views, no FiLM, no diffusion). No memory module
is present in the checkpoint.
| Setting | Value |
|---|---|
| Checkpoint step | 150000 |
| Memory | none (use_memory off) |
| Dataset | MIKASARoboVLAEpisodicDataset, mikasa_five |
| Training environments | ShellGamePush-VLA-v0, InterceptMedium-VLA-v0, RememberColor5-VLA-v0, TakeItBack-VLA-v0, RememberShapeAndColor3x3-VLA-v0 |
| GPUs | 8 x NVIDIA A100 80GB |
| Batch size | 4 per GPU |
| LoRA rank | 32 |
| Learning rate | 5e-4, constant for 100000 steps, then decayed |
| Image augmentation | on |
The learning rate schedule is the difference to keep in mind when reading this model as a control: the memory checkpoints use a cosine schedule with 2000 warmup steps and a minimum ratio of 0.1, while this run uses the constant-then-decay schedule of the original OpenVLA-OFT recipe. Everything else, including the dataloader, the mixture and the number of steps, is identical.
The episodic dataloader feeds batch_size independent streams, each replaying whole episodes in
order, and marks every step with is_first and is_last. It is used here as well, so that the
control sees exactly the same frame ordering as the memory arms; without a memory state, that
ordering only affects which frames land in a batch. Action normalisation statistics (q01/q99)
are computed jointly over all five environments and stored under the mikasa_combined key in
dataset_statistics.json.
Protocol: each environment is evaluated independently, 100 episodes, starting seed 4242424242,
metric success_once.
Both inference modes are reported. Receding horizon queries the model at every environment step and executes only the first of the 8 predicted actions; open loop executes the whole predicted chunk before querying again. Receding horizon costs 8 times more forward passes and is the mode used for the numbers quoted in the memory checkpoints' cards.
Only the first five rows are training environments. Everything else is zero-shot transfer to environments this model never saw during fine-tuning.
| Environment | Receding horizon | Open loop |
|---|---|---|
ShellGamePush-VLA-v0 (train) | 0.33 | 0.90 |
InterceptMedium-VLA-v0 (train) | 0.53 | 0.39 |
RememberColor5-VLA-v0 (train) | 0.23 | 0.09 |
TakeItBack-VLA-v0 (train) | 0.98 | 0.87 |
RememberShapeAndColor3x3-VLA-v0 (train) | 0.11 | 0.13 |
ShellGameTouch-VLA-v0 | 0.00 | 0.00 |
ShellGamePick-VLA-v0 | 0.01 | 0.00 |
InterceptSlow-VLA-v0 | 0.06 | 0.04 |
InterceptFast-VLA-v0 | 0.24 | 0.33 |
InterceptGrabSlow-VLA-v0 | 0.00 | 0.00 |
InterceptGrabMedium-VLA-v0 | 0.00 | 0.00 |
InterceptGrabFast-VLA-v0 | 0.00 | 0.00 |
RotateLenientPos-VLA-v0 | 0.08 | 0.04 |
RotateLenientPosNeg-VLA-v0 | 0.07 | 0.08 |
RotateStrictPos-VLA-v0 | 0.03 | 0.04 |
RotateStrictPosNeg-VLA-v0 | 0.05 | 0.03 |
RememberColor3-VLA-v0 | 0.25 | 0.19 |
RememberColor9-VLA-v0 | 0.07 | 0.11 |
RememberShape3-VLA-v0 | 0.17 | 0.08 |
RememberShape5-VLA-v0 | 0.15 | 0.11 |
RememberShape9-VLA-v0 | 0.11 | 0.11 |
RememberShapeAndColor3x2-VLA-v0 | 0.12 | 0.09 |
RememberShapeAndColor5x3-VLA-v0 | 0.08 | 0.06 |
| File | Purpose |
|---|---|
model-0000*-of-00004.safetensors, model.safetensors.index.json | merged vision-language backbone |
lora_adapter/ | LoRA adapter as saved by PEFT, before merging |
action_head--150000_checkpoint.pt | L1 regression action head |
proprio_projector--150000_checkpoint.pt | proprioception projector |
dataset_statistics.json | action normalisation statistics, key mikasa_combined |
configuration_prismatic.py, modeling_prismatic.py, processing_prismatic.py | custom model code |
There is no memory_module--150000_checkpoint.pt and no memory_meta.json here: that is the
point of this checkpoint. Optimizer and scheduler state are not published for this run.
huggingface-cli download mu-vla/mu-vla-openvla-oft-mikasa-robo-5-tasks-no-memory \
--local-dir ./openvla-oft-mikasa-no-memory
The checkpoint needs the mu-VLA code, which depends on a forked transformers, and on
MIKASA-Robo for the environments. Point the
evaluation script at the downloaded directory; with no memory_meta.json present it runs as plain
OpenVLA-OFT. See CognitiveAISystems/muVLA for setup
and the exact command line.
mu-VLA is a fork of OpenVLA-OFT by Moo Jin Kim, Chelsea Finn and Percy Liang (arXiv:2502.19645), which in turn builds on OpenVLA. The recurrent memory module, the episodic dataloaders and the MIKASA-Robo integration are the contribution of this work. Released under the MIT license, following upstream.