vjepa2-ac-vitg-GGUF

V-JEPA 2-AC ViT-g (action-conditioned world model) — GGUF for jepa.cpp

Meta's V-JEPA 2-AC — the action-conditioned world model behind the paper's zero-shot robot planning — converted to GGUF for jepa.cpp, a ggml C/C++ engine that runs it on a plain CPU with no Python and no PyTorch. One bundle: the frozen ViT-g/16 encoder from vjepa2-ac-vitg.pt plus the 24-layer, 1024-dim predictor that takes a 7-d end-effector action and a 7-d pose per frame and predicts the next frame's latents, block-causally over frames. jepa_ac_rollout scores K candidate action sequences in one graph per step, and jepa_ac_energy is the L1 planning energy a CEM planner minimises.

Note on the encoder: it is not facebook/vjepa2-vitg-fpc64-256. Meta's AC checkpoint carries its own frozen ViT-g, which agrees with the HF release only to cosine ~0.998 per tensor; encoder and target_encoder inside it are bit-identical. This bundle ships the checkpoint's own, which is what vjepa2_ac_vit_giant loads.

1317 M parameters; D = 1408, 40 layers, 22 heads, patch 16, tubelet 2, 256x256. Everything the engine needs — dimensions, positional scheme, preprocessing recipe, and class labels where there are any — travels inside the file, so inference needs one binary and one GGUF and nothing else.

Run it

git clone --recursive https://github.com/aselimc/jepa.cpp && cd jepa.cpp
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release && cmake --build build -j
hf download jepacpp/vjepa2-ac-vitg-GGUF vjepa2-ac-vitg-f16.gguf --local-dir models/gguf

# encode a frame, roll 4 candidate action sequences out 2 steps, score them against a goal
build/jepa-worldmodel --ac -m vjepa2-ac-vitg-f16.gguf --image ctx.png --goal goal.png \
    --actions-npy actions.npy   # float32 [K, H, 7]

--pool selects mean, cls, lewm or none (the full token map); -o writes a .npy. scripts/download_models.sh fetches whole sets at once. The C API is one header, include/jepa.h — full reference on the C API page.

Files

_{ᵖ tests/test-parity on the CPU backend, stored reference input, 32 threads, worst sample — docs/parity.md. ᵈ scripts/gguf_dequant_selftest.py: the dequantized weights through the numpy reference graph at f32 activations, so the figure is the weight error alone — docs/quantization.md. cos mean is the mean per-token cosine of last_hidden_state, worst its single worst token.}

Tiers. exact — reproduces the PyTorch reference to the printed precision on the CPU. parity — passes its family's test-parity thresholds. advisory — below 8 bits per weight, which is not a parity configuration: the results are reported, only the derived tensors and the top-1 are gated. Which file to ship: Accuracy → which dtype. vjepa2-ac-vitg-q8_0.gguf multi-step rollouts. vjepa2-ac-vitg-q4_0.gguf multi-step rollouts. vjepa2-ac-vitg-q4_k.gguf multi-step rollouts.

Full checksums:

234b7e775da2ff28e29d6b12a48fd28f6b0e888b1854581a684dc8e8e84848bb  vjepa2-ac-vitg-f32.gguf
d57cc1760f90f779eba12d90d04b5028334a95f9b6960da6a98239b9a6f2441c  vjepa2-ac-vitg-f16.gguf
3dffe7130d7f7efd0805d2d714e72a232632077cc59a9036b7896435f6c40ef6  vjepa2-ac-vitg-q8_0.gguf
554f3a9f293a2b80d3e0f9dd40f033e4ca1a291640712606a727a419ee0f4425  vjepa2-ac-vitg-q4_0.gguf
0d2fb4e022e2d1ca9a3aaf93d5ae3ad51b1d41244b04f634561ba2ac48a5dfbc  vjepa2-ac-vitg-q4_k.gguf

Verify a download with sha256sum -c. The other types jepa-quantize can produce (q4_1, q5_0, q5_1, q5_k, q6_k, measured in quantization) are not published here; make them locally with build/jepa-quantize vjepa2-ac-vitg-f16.gguf out.gguf q6_k -t 32.

Measured

Every figure below is read from a committed artifact of jepa.cpp 00bfd4e by scripts/hf_publish.py — parity, quantization, accuracy, performance and tests/results/*.json.

Speed — the encoder graph at f16 on 32 threads (AMD Ryzen Threadripper PRO 7995WX 96-Cores): 2228 ms per 16-frame clip; 15747 ms per 64-frame clip. The same shape on NVIDIA RTX 4500 Ada Generation: 27.8 ms. Peak RSS at f16: 3092 MiB.

Plan with f16. A rollout compounds — step h's input is step h-1's output — so the worst predicted token of a 2-step rollout falls from 0.9925 at f16 to 0.9368 at q8_0 to 0.5429 at q4_k, and at q4_k on a GPU the planning energy misranks the candidates and the model picks a different action. The encoder half of the bundle passes at every tier on both backends. At f32 on the CPU the predictor is exact to cosine 1.0000000 against Meta's own world-model dump, and K candidates batched on the graph's batch axis are bit-identical to K sequential rollouts, and jepa_ac_plan reproduces Meta's own CEM planner to 3e-08 on its own recorded random draws. Full tables in parity.

On -q4_k.gguf: K-quants need ne[0] % 256 == 0, and the encoder's width is 1408 = 5.5 x 256, so every [*, 1408]-row matrix falls back to q4_0. In THIS bundle the 24 predictor blocks are 1024-wide and keep the K-quant, so the mix is 121 q4_0 against 137 q4_K and the file reports general.file_type = 12 (q4_k). (The encoder-only vjepa2-vitg-fpc64-256-q4_k.gguf goes the other way: 158 q4_0 / 52 q4_K, reported as q4_0.) jepa-info lists the real mix.

Source, licence and attribution

Converted from dl.fbaipublicfiles.com/vjepa2/vjepa2-ac-vitg.pt.

MIT. The checkpoint is published by Meta AI (FAIR) at https://dl.fbaipublicfiles.com/vjepa2/vjepa2-ac-vitg.pt; the licence is the LICENSE of facebookresearch/vjepa2 (Copyright (c) Meta Platforms, Inc. and affiliates). No gating, no acceptable-use policy. These GGUF files are the same weights re-serialised, quantized where the file name says so. Cite the V-JEPA 2 paper.

The licence travels inside every GGUF as general.license and the origin as general.source_url; build/jepa-info <file> --kv prints them. jepa.cpp's own code is MIT.

Conversion

Produced by jepa.cpp 00bfd4e:

scripts/download_models.sh --convert vjepa2-ac
python scripts/convert.py --family vjepa2_ac --src models/vjepa2_ac/vjepa2-ac-vitg.pt --out models/gguf/vjepa2-ac-vitg-f16.gguf --ftype f16
#   ... and again with --ftype f32 --out models/gguf/vjepa2-ac-vitg-f32.gguf for the f32 file

for q in q8_0 q4_0 q4_k; do
  build/jepa-quantize models/gguf/vjepa2-ac-vitg-f16.gguf \
      models/gguf/vjepa2-ac-vitg-$q.gguf $q -t 32
done

jepa-quantize re-types only the 2-D attention / FFN / projection / classifier matrices; patch embeddings, position tables, norms and biases keep the source type. The rules are in docs/gguf-schema.md.

Links

• Code: https://github.com/aselimc/jepa.cpp
• Documentation: https://aselimc.github.io/jepa.cpp/
• Parity fixtures: https://huggingface.co/datasets/jepacpp/jepa.cpp-fixtures
• All jepa.cpp GGUFs: https://huggingface.co/jepacpp