vla-adapter-libero-gguf

VLA-Adapter - LIBERO-Object (GGUF for vla.cpp)

GGUF conversion of VLA-Adapter/LIBERO-Object-Pro for inference with vla.cpp, a lightweight C++ inference engine for Vision-Language-Action models built on top of llama.cpp.

VLA-Adapter is a tiny-scale VLA: a Prismatic-style VLM with a fused DINOv2 + SigLIP-SO400M vision backbone (224 px) over a Qwen2.5-0.5B language backbone, coupled to a Bridge-Attention policy head that fuses VLM conditions with the action stream to predict an 8-step action chunk. Despite the 0.5B backbone it is competitive with much larger open VLAs. The vision tower is baked into the combined GGUF, so no separate mmproj file is needed.

Files

Usage

Build vla-server from the vla.cpp repo:

# Build vla-server (CPU build shown; use a CUDA build for GPU inference)
git clone https://github.com/VinRobotics/vla.cpp && cd vla.cpp
bash ./patches/patch.sh                        # fetch + patch llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j"$(nproc)"

Then serve and drive an episode:

# Terminal 1 - serve. No mmproj argument (vision is baked into the combined GGUF).
./build/vla-server --bind tcp://*:5566 \
    vla-adapter-libero-object.gguf

# Terminal 2 - drive a LIBERO episode (inside the LIBERO uv venv)
eval/sim/libero/libero_uv/.venv/bin/python eval/client/run_sim_client_direct.py \
    --arch vla_adapter \
    --task libero_object --task-id 0 --n-episodes 10 \
    --n-action-steps 8 \
    --vla-addr tcp://localhost:5566

vla_adapter runs at 224 px with an 8-step action chunk (--n-action-steps 8) and proprio state dim 8. The tokenizer auto-loads from the base checkpoint; pass --tokenizer /path/to/LIBERO-Object-Pro to load it offline.

Benchmark

Smoke test - libero_object task 0, 10 episodes, vla-server + run_sim_client_direct.py:

The upstream PyTorch checkpoint reports ≈99.6% on the full LIBERO-Object suite; the full vla.cpp sweep (10 tasks × 20 episodes = 200 episodes) will be filled in once measured on reference hardware.

License

Weights follow the upstream license of VLA-Adapter/LIBERO-Object-Pro (MIT). The vla.cpp conversion tooling and inference engine are MIT-licensed.