NanoVDR-D-Fast-Qwen3VL8B-4096

<p align="center"> <img width="440" src="https://huggingface.co/nanovdr/NanoVDR-Q-DistilBERT-Qwen3VL2B-2048-ML/resolve/main/banner.png" alt="NanoVDR"/> </p> <h3 align="center">NanoVDR-D-Fast-Qwen3VL8B-4096: the document tower, 3x fewer visual tokens</h3> <p align="center"> <a href="https://ryenhails.github.io/NanoVDR/">Project page</a> &nbsp;|&nbsp; <a href="https://huggingface.co/nanovdr">Models</a> &nbsp;|&nbsp; <a href="https://github.com/Ryenhails/NanoVDR">Code</a> &nbsp;|&nbsp; <a href="https://huggingface.co/spaces/nanovdr/NanoVDR-Demo">Demo</a> &nbsp;|&nbsp; <a href="https://huggingface.co/spaces/nanovdr/distilling-the-document-tower">Write-up</a> &nbsp;|&nbsp; <a href="https://arxiv.org/abs/2608.10636">Paper</a> </p>

---

Renamed. This model was published as nanovdr/NanoVDR-D-Fast. Old links redirect. The name now states the tower (D), the tile budget, the teacher it was distilled from and the output width: pair it with any tower whose teacher and width match, in this case NanoVDR-Q-DistilBERT-Qwen3VL8B-4096-ML.

A 457M encoder that turns a document page image into one 4096-d vector. It is distilled from a frozen 8B vision-language teacher (Qwen3-VL-Embedding-8B) by direct representation alignment: the student is trained only to reproduce the teacher's page embedding, with no relevance labels, no negative mining and no contrastive term.

Every other NanoVDR release replaces the query side and still needs the teacher to build the index. This one replaces the document side, so paired with a NanoVDR query tower the teacher is not needed at all, at indexing time or at query time.

How it works

page ──► dynamic tiling ──► InternViT-300M-448 ──► linear ──► ModernBERT-base ──► mean pool ──► Linear(768→4096) ──► L2-norm
         (≤2 tiles +          per-tile patch        1024→768    bidirectional,
          thumbnail)          tokens, concatenated              no text tokens

The tiling rule matches the page's aspect ratio to a grid at the encoder's native 448px resolution and appends a thumbnail for global context, so small text and table cells survive without the patch sequence outrunning ModernBERT's 8192-token window.

Usage

Both towers are called the same way. The document tower needs sentence-transformers>=5.4, which is the release that routes image inputs to the model.

from PIL import Image
from sentence_transformers import SentenceTransformer

docs = SentenceTransformer("nanovdr/NanoVDR-D-Fast-Qwen3VL8B-4096", trust_remote_code=True)
queries = SentenceTransformer("nanovdr/NanoVDR-Q-DistilBERT-Qwen3VL8B-4096-ML")

doc_emb = docs.encode([Image.open("page1.png"), Image.open("page2.png")])
query_emb = queries.encode(["what was the revenue growth in Q3 2024?"])

scores = query_emb @ doc_emb.T          # (n_queries, n_pages)

Through transformers instead, which has no version floor:

import torch
from transformers import AutoModel, AutoImageProcessor

model = AutoModel.from_pretrained("nanovdr/NanoVDR-D-Fast-Qwen3VL8B-4096", trust_remote_code=True).eval()
processor = AutoImageProcessor.from_pretrained("nanovdr/NanoVDR-D-Fast-Qwen3VL8B-4096", trust_remote_code=True)

inputs = processor(images=pages, return_tensors="pt")     # pixel_values + tile_mask
with torch.no_grad():
    doc_emb = model(**inputs).embedding                   # (N, 4096), L2-normalised

processor is a NanoVDRDocImageProcessor. It tiles the page and returns the tile_mask the model needs next to pixel_values, so a stock single-view image processor is not a substitute: it silently costs several NDCG points, and the model warns if it is handed one. model.encode(pages, processor, batch_size=8) does the same thing with batching.

On Ampere or newer, switch the fused kernels on:

model = AutoModel.from_pretrained(
    "nanovdr/NanoVDR-D-Fast-Qwen3VL8B-4096",
    trust_remote_code=True,
    use_flash_attn=True,                       # InternViT
    text_attn_implementation="flash_attention_2",
).to("cuda", torch.bfloat16)

Weights ship in float32 so the model loads on any device; cast to bfloat16 for deployment.

Any half of the teacher, swapped

Both towers are trained to land in the same frozen teacher's embedding space, so either one is a drop-in replacement for the corresponding half of that teacher. All four combinations are valid retrieval systems, and they trade quality against what you no longer have to run:

_{Average NDCG@5 over ViDoRe v1+v2+v3, with NanoVDR-D-HiRes-Qwen3VL8B-4096 and NanoVDR-Q-DistilBERT-Qwen3VL8B-4096-ML against Qwen3-VL-Embedding-8B.}

The pairing rule follows from the naming: the teacher and the dimension must match, and any cell of the matrix is then available. The teacher fixes the space; the dimension fixes which width of it is targeted, since a Matryoshka teacher can be aligned to at several widths.

Results

NDCG@5 on all 22 datasets of ViDoRe v1, v2 and v3. Every baseline was reproduced locally under one protocol rather than quoted from its own paper, so these numbers are comparable to each other and not to published tables.

End to end, paired with the 70M text-only query tower distilled from the same teacher, 527M total, with nothing multi-billion running at deployment:

That is 84.4% of the teacher, and a lead of 6.97 average points over the strongest sub-1B baseline we reproduced.

Deployment cost

Single H200, batch size 8, bfloat16. "Score 10K" is one query against 10 000 candidates on a single CPU thread.

One vector per page instead of a token set is what buys the last two rows: a 15.6x smaller index and two orders of magnitude less scoring work than the multi-vector retrievers in the same parameter class. Fast also indexes 2.7x faster than HiRes, which costs it 1.76 average points and 3.41 on v3.

Training

Teacher targets are cached once before training, so the document and query towers train independently and in parallel.

Limitations

• Bounded by the teacher. The student only reproduces Qwen3-VL-Embedding-8B's embedding space; any systematic weakness of that teacher carries over, and nothing in the objective lets the student exceed it.
• Fixed, small tile budget. Two tiles plus a thumbnail is not enough for the densest pages; on ViDoRe v3 this costs 3.41 points against HiRes. Tiling also adapts to page aspect ratio but not to page content.
• Evaluation is ViDoRe-only. Three difficulty levels, six languages, eight professional domains, but not in-house enterprise layouts, scanned or OCR-degraded pages, or production query distributions.
• Uncompressed index. One 4096-d float32 vector per page. Quantisation and product quantisation are untested here, so 16.4 GB per million is an upper bound.

Citation

This model is described in DistilVDR: A Compact End-to-End Visual Document Retriever via Dual-Student Distillation.

@article{distilvdr2026,
  title   = {DistilVDR: A Compact End-to-End Visual Document Retriever
             via Dual-Student Distillation},
  author  = {Liu, Zhuchenyang and Wang, Ziyi and Zhang, Yao and Xiao, Yu},
  journal = {arXiv preprint arXiv:2608.10636},
  year    = {2026}
}

Distilling the Document Tower is the write-up behind this model: the teacher and data choices, the architecture, all fourteen ablations, and the five things that did not work.

License

MIT. The visual and text backbones follow the licences of InternViT-300M-448px-V2_5 and ModernBERT-base.