Downloads · 30 days
4.8K
98% of all-time downloads
canvit/canvitb16-add-vpe-pretrain-g128px-s512px-in21k-dv3b16-2026-02-02-nnx
canvitb16-add-vpe-pretrain-g128px-s512px-in21k-dv3b16-2026-02-02-nnx is a image feature extraction model from canvit. Use it for the image feature extraction task on the model card, and read the license before you ship it in a product. It is set up for canvit-nnx. The card lists the license as mit.
canvit/canvitb16-add-vpe-pretrain-g128px-s512px-in21k-dv3b16-2026-02-02 in the checkpoint format of canvit-nnx, for JAX / Flax NNX.
Downloads · 30 days
4.8K
98% of all-time downloads
All-time downloads
4.9K
Public
Parameters
98.1M
773 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors392 MB · 100%
From the Hugging Face model README
canvit/canvitb16-add-vpe-pretrain-g128px-s512px-in21k-dv3b16-2026-02-02 in the checkpoint format of canvit-nnx, for JAX / Flax NNX.
CanViT, the Canvas Vision Transformer, is an active-vision foundation model: it sees a scene through a sequence of glimpses and remembers it on a scene-wide canvas.
Paper (NeurIPS 2026) · arXiv · Code · Project page · All checkpoints
uv add canvit-nnx
import jax.numpy as jnp
from PIL import Image
from canvit_nnx import CanViTForPretraining, Viewpoint, sample_at_viewpoint
from canvit_nnx.preprocess import preprocess
scene = jnp.asarray(preprocess(512)(Image.open("scene.jpg").convert("RGB")))[None]
model = CanViTForPretraining.from_pretrained("canvit/canvitb16-add-vpe-pretrain-g128px-s512px-in21k-dv3b16-2026-02-02-nnx")
model.eval()
state = model.init_state(batch_size=1, canvas_grid_size=32)
viewpoint = Viewpoint.full_scene(batch_size=1)
glimpse = sample_at_viewpoint(spatial=scene, viewpoint=viewpoint, glimpse_size_px=128)
output = model(glimpse=glimpse, state=state, viewpoint=viewpoint)
state = output.state
| Backend | JAX / Flax NNX |
| Model class | CanViTForPretraining |
| Converted from | canvit/canvitb16-add-vpe-pretrain-g128px-s512px-in21k-dv3b16-2026-02-02 at 13da0ca2725f569bf486e1efdcaaac32f0549bf9 |
| Conversion checked at | 512 px scenes, 128 px glimpses, 32 × 32 canvas |
MIT. CanViT is pretrained by distillation from DINOv3, which is released under the DINOv3 License.
@inproceedings{berreby2026canvit,
title={{CanViT}: Toward Active-Vision Foundation Models},
author={Berreby, Yoha{\"i}-Eliel and Du, Sabrina and Durand, Audrey and Krishna, B. Suresh},
booktitle={The Fortieth Annual Conference on Neural Information Processing Systems},
year={2026},
url={https://openreview.net/forum?id=cnV9GA4pKU},
eprint={2603.22570},
archivePrefix={arXiv},
primaryClass={cs.CV}
}