vjepa2-ac-vitg-fpc64-256-droid-tt

<!-- Weights repo, not a tt-model bundle. This card intentionally carries no tt-model-cache or thin tags. A tt-model bundle for the TTNN port has not been published yet; when one is, its manifest's weights.repo_id will point here. base_model note: Meta never published the action-conditioned checkpoint on the Hub (the old value, facebook/vjepa2-ac-vitg-fpc64-256-droid, does not exist, so the Hub ignored it). facebook/vjepa2-vitg-fpc64-256 is the ViT-g/256 encoder (`vitg.pt`) that V-JEPA 2-AC was post-trained from, per configs/train/vitg16/droid-256px-8f.yaml. -->

vjepa2-ac-vitg-fpc64-256-droid-tt

This is a stripped, bf16, inference-only copy of Meta's V-JEPA 2-AC ViT-giant action-conditioned checkpoint (vjepa2-ac-vitg.pt). V-JEPA 2-AC was post-trained on the DROID robot-trajectory dataset and runs at 8 frames @ 256px. The file is here so the Tenstorrent Blackhole / TTNN port at tsingletaryTT/tt-vjepa2 has a download of about 2.6 GB instead of about 11 GB.

This is a weights repo only. It holds one checkpoint file. It is not a tt-model package, and tt model pull / tt-model pull cannot install it. To run it on Tenstorrent hardware today, use the port's GitHub repo directly; no installable tt-model bundle has been published yet.

What's in the file

Same weights, smaller file. The model is not fine-tuned or modified. It is Meta's official checkpoint with the following changes:

• dropped the optimizer state, gradient scaler, and momentum target-encoder copy (none of these are used at inference)
• cast every remaining tensor from fp32 to bf16

The transform is scripts/strip_checkpoint.py. You can run it yourself on the official checkpoint to reproduce this file.

What V-JEPA 2-AC is

V-JEPA 2-AC is a latent, action-conditioned world model. It is post-trained from V-JEPA 2's self-supervised video encoder on a small amount of robot interaction data. It solves manipulation tasks without task-specific training or environment calibration. It predicts embeddings, never pixels. For details, see the paper and the official repo.

Using this checkpoint

import torch

ckpt = torch.load("vjepa2-ac-vitg.inference.pt", map_location="cpu", weights_only=True)
encoder_state_dict = ckpt["encoder"]      # ViT-g encoder, 40 blocks
predictor_state_dict = ckpt["predictor"]  # AC predictor, 24 blocks

To download it programmatically:

from huggingface_hub import hf_hub_download

path = hf_hub_download(
    "episod/vjepa2-ac-vitg-fpc64-256-droid-tt",
    "vjepa2-ac-vitg.inference.pt",
    revision="be4912a0f92001a0ffab22a9dc014d8e96c3f3a5",  # pin, if you need reproducibility
)

The state dicts load directly into VJEPA2Encoder.from_state_dict and VJEPA2Predictor.from_state_dict in the TTNN port. They also load into Meta's reference VisionTransformer and VisionTransformerPredictorAC. Cast them to fp32 first if you want the reference's numerics.

Measured with this checkpoint

The numbers below describe the TTNN port running these weights. They don't describe the checkpoint on its own. For the full table, methodology, and caveats, see the tt-vjepa2 README and MODEL_CARD.md.

• Correctness, as PCC against the reference PyTorch implementation (fp32, these weights):
  • encoder 0.9970
  • predictor 0.9972
  • bar ≥ 0.995
• Speed, 8 frames @ 256px, single Blackhole chip, TTNN bf16-mixed, traced replay:
  • 120.7 ms/forward, 66.3 input-frames/s
  • 35.6× faster than the same host's CPU (4293.2 ms/forward, reference PyTorch fp32 eager, AMD Ryzen 7 9700X)
  • No GPU was available to compare against.
• IntPhys 2, public Main split (506 pairs):
  • 0.613 pairwise accuracy (chance 0.5)
  • by difficulty: Easy 0.731, Medium 0.605, Hard 0.583
  • The solidity condition alone scores at chance (0.507).
  • Scoring is violation-of-expectation, adapted to this action-conditioned predictor by a zero-action rollout. That adaptation is a real methodological compromise.

Not evaluated: MVPBench, CausalVQA, Something-Something v2, Epic-Kitchens-100, or any real-robot, LIBERO, or SimplerEnv success-rate trial.

Limitations

• bf16 storage is lossy compared with the fp32 original. Every PCC figure above was measured from this bf16 file. The TTNN port keeps LayerNorm and the residual stream in fp32 for that reason.
• Only the context encoder is kept, not the momentum target_encoder. That is enough for inference and planning, but not for resuming training.
• The checkpoint is 256px, 8-frame (DROID) only. For other resolutions, use Meta's other releases.

License

These weights are Meta's, redistributed in modified form (stripped and cast to bf16) under the MIT License that facebookresearch/vjepa2 releases them under. Copyright (c) Meta Platforms, Inc. and affiliates. See LICENSE for the full notice.

• Meta distributes the V-JEPA 2-AC checkpoint only through the GitHub repo's download link. That repo's license is MIT. The README's License section names only three data-aug and worker source files as Apache-2.0.
• Meta's Hub-hosted V-JEPA 2 encoder repos, such as facebook/vjepa2-vitg-fpc64-256 (listed as base_model above), are labeled apache-2.0 on the Hub. Both licenses are permissive and allow redistribution and commercial use.
• The TTNN port code at tsingletaryTT/tt-vjepa2 is separately MIT, Copyright (c) 2026 Taylor Singletary.

Changelog

Citation

@article{assran2025vjepa2,
  title={V-JEPA~2: Self-Supervised Video Models Enable Understanding, Prediction and Planning},
  author={Assran, Mahmoud and Bardes, Adrien and Fan, David and Garrido, Quentin and Howes, Russell and
Komeili, Mojtaba and Muckley, Matthew and Rizvi, Ammar and Roberts, Claire and Sinha, Koustuv and Zholus, Artem and
Arnaud, Sergio and Gejji, Abha and Martin, Ada and Robert Hogan, Francois and Dugas, Daniel and
Bojanowski, Piotr and Khalidov, Vasil and Labatut, Patrick and Massa, Francisco and Szafraniec, Marc and
Krishnakumar, Kapil and Li, Yong and Ma, Xiaodong and Chandar, Sarath and Meier, Franziska and LeCun, Yann and
Rabbat, Michael and Ballas, Nicolas},
  journal={arXiv preprint arXiv:2506.09985},
  year={2025}
}