VERA

sizhe-lester-li/VERA

VERA is a robotics model from sizhe-lester-li. Use it for the robotics task on the model card, and read the license before you ship it in a product. The card lists the license as other.

Hosted weights for VERA — Turning Video Models into Generalist Robot Policies — a two-stage system: a video planner that "dreams" the future, and a Jacobian inverse-dynamics model (IDM) that turns the dream into robot…

Downloads · 30 days

Access

Public

Updated Sep 5, 2026

Repo size

159 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.ckpt53.1 GB · 59%

At a glance

Task: Robotics
License: other
Access: Public
Created: Jun 22, 2026
Updated: Sep 5, 2026
SHA: 99fdfe71

Model card

From the Hugging Face model README

VERA checkpoints

Hosted weights for VERA — Turning Video Models into Generalist Robot Policies — a two-stage system: a video planner that "dreams" the future, and a Jacobian inverse-dynamics model (IDM) that turns the dream into robot actions. VERA hosts only the trained artifacts; frozen upstream pieces (Wan2.1 text-enc/VAE, VGGT) are pulled from their original homes and filtered out at inference.

Code + usage: see the VERA GitHub repo.

Release groups

1 · Panda-sim (MimicGen)

dir	what
`mimicgen-wan-1.3b/`	MimicGen-specialist WAN video planner (1.3B, t2v→v2v), DiT-only bf16 (~2.8 GB, de-bloated from 17.5 GB) + `flow_decoder.ckpt` + `algo_config.yaml`. Pull `Wan-AI/Wan2.1-T2V-1.3B` base upstream.
`idm-mimicgen-285ouq1q/`	Jacobian IDM (run `285ouq1q`, VGGT-1B) used for the reported MimicGen `stack_d0` evaluations; pairs with `mimicgen-wan-1.3b/`.

Pair these two + cotracker + the gated/adaptive gripper controller to run stack_d0.

2 · PushT — the two-stage DFoT→Jacobian pipeline

dir	what
`pusht-dfot/`	DFoT video planner (run `dvxixf6d`) — a U-Net3D flow predictor (~2.4M params), not WAN. + `run_config.yaml`.
`pusht-idm/`	PushT Jacobian IDM (run `j1j59qzz`, ~34.8M params, 2-DOF) + `config.yaml`.

3 · OMNI — the cross-embodiment WAN planner

dir	what
`omni-wan/`	OMNI WAN planner (14B i2v→v2v, `combined_4env`), DiT-only bf16 (~33 GB). Pull `Wan-AI/Wan2.1-I2V-14B-480P` base upstream.
`idm-mimicgen/`	MimicGen IDM (`x21o0cwe`) — the OMNI-default pairing.
`idm-droid/`	DROID FR3 IDM (`7wohna95`, SE3-delta 7-DOF).

4 · DROID — real-robot WAN planner

dir	what
`wan-droid-14b/`	DROID-specialist WAN video planner (14B i2v→v2v, 29 context frames → 24-frame chunks, 3-view canvas), DiT-only bf16 (~31 GB) + `algo_config.yaml`. Pull `Wan-AI/Wan2.1-I2V-14B-480P` base upstream. Used by `examples/droid_generation.ipynb` in the GitHub repo — language-conditioned video generation from real context frames (no server or robot needed).
`droid-demo-clips/`	Real DROID context clips (~100 MB) loaded by the generation notebook.

Licenses (per checkpoint)

VERA's own contributions (WAN video planners, Jacobian IDM heads, DFoT planner) are Apache-2.0. Two IDM checkpoints additionally bundle the VGGT-1B backbone weights, which are CC-BY-NC-4.0 — so those combined artifacts are non-commercial.

dir	license	why
`wan-droid-14b/`, `omni-wan/`, `mimicgen-wan-1.3b/`	Apache-2.0	WAN planners; inherit Wan2.1 Apache-2.0
`pusht-dfot/`, `pusht-idm/`	Apache-2.0	authors' weights; VGGT not bundled
`idm-mimicgen/`	Apache-2.0	authors' IDM head only; pulls VGGT-1B upstream at runtime (itself CC-BY-NC-4.0)
`idm-droid/`, `idm-mimicgen-285ouq1q/`	CC-BY-NC-4.0	bundle VGGT-1B weights in-checkpoint

Frozen upstream pieces (Wan2.1 text-enc/VAE, VGGT-1B) keep their original licenses.

Training data — `pusht-packed/` and `pusht-noise-packed/`

Two packed PushT training sets, same on-disk format (one NPZ per episode with 504×504 JPEG frames, MegaFlow optical flow qint8_zstd_npz-quantized, and per-episode state, plus index.json / format_manifest.json; per-folder provenance in provenance.json, file hashes in SHA256SUMS.txt):

Folder	Episodes	Size	Content	Actions
`pusht-packed/`	206	~1.9 GB	human-teleop episodes — the Jacobian IDM training set (`config_pusht_vggt_fusion_jacobian`)	command actions from `pusht-replay/`, the original `pusht_cchi_v7_replay.zarr` archive
`pusht-noise-packed/`	18,685	~61 GB	uniform-random exploration episodes — IDM pretraining / data-efficiency ablations	self-contained: state deltas from the embedded `traj_state` arrays (no external file)

hf download sizhe-lester-li/VERA --include "pusht-packed/*" --local-dir ./vera-ckpts
hf download sizhe-lester-li/VERA --include "pusht-noise-packed/*" --local-dir ./vera-ckpts   # ~61 GB

The standard pack also needs the original command-action replay, now mirrored in pusht-replay/ (31 MB, 206 episodes). Download, verify and extract:

hf download sizhe-lester-li/VERA --include "pusht-replay/*" --local-dir ./vera-data
(cd vera-data/pusht-replay && sha256sum -c SHA256SUMS.txt)
unzip -n vera-data/pusht-replay/pusht.zip -d data

Set dataset.pusht_zarr_root to the absolute path of data/pusht.

Setup + training: TRAINING.md — "Get the training data (PushT)".

License note: the pusht-packed source episodes are the MIT-licensed PushT data from the Diffusion Policy benchmark; the pusht-noise-packed source trajectories are the PushT exploration data from DINO-WM (Zhou et al., 2024). Both were re-rendered with gym-pusht; optical-flow annotations were added with MegaFlow. Regeneration reference: scripts/data/pack_pusht.py in the VERA repo (walkthrough: docs/DATA_GENERATION.md).

Task: Robotics
License: other
Created: Jun 22, 2026
Updated: Sep 5, 2026