VisionPsy-Nano-460M-Flash-GGUFs

VisionPsy-Nano-460M-Flash-GGUFs

VisionPsy-Nano-460M-Flash-GGUFs provides GGUF weights of VisionPsy-Nano-460M-Flash for fast, fully on-device multimodal inference with our patched llama.cpp fork (not stock upstream). Flash is the efficiency-optimized sibling of VisionPsy-Nano-460M: same ~460M nanoVLM architecture, but it processes each image with far fewer visual tokens (as few as 64 at 512x512), so it responds an order of magnitude faster on real phones while giving up only a small amount of accuracy.

Unquantized FP32 and BF16 GGUFs are included alongside a family of quantization formats, from near-lossless 8-bit through a high-quality 5-bit option down to ultra-compact 3-bit. All sub-8-bit variants (except the legacy Q4_0) use importance-matrix (imatrix) calibration. If VisionPsy-Nano-460M-GGUF is our best-quality ~0.5B GGUF, Flash-GGUF is the one you ship when time-to-first-token, battery, and RAM are the binding constraints.

Each GGUF is a vision-language model: alongside the text weights (*.gguf) you also need the multimodal projector (mmproj-*.gguf, the SigLIP2 vision encoder) to run image inputs. See Usage.

^* References to Tether AI Research are references to Tether Data, S.A. de C.V.

---

The headline: finishes faster on real phones

Flash is an order of magnitude faster on real phones. By leveraging less visual tokens, Flash reaches the first token 10-25x faster than nanoVLM-460M and SmolVLM2-500M and consistently faster than LFM2.5-VL-450M and Qwen3.5-0.8B on Pixel 9, Galaxy S23, Galaxy S25, and iPhone 15, while keeping ~98% of the full model's quality (61.1 vs 62.3 normalized).

---

Available Files

All published files are produced with llama.cpp. The FP32 and BF16 GGUF files are unquantized (no quantization applied) and serve as the in-harness reference. Q8_0 does not use imatrix calibration (imatrix provides no measurable benefit at 8-bit), and Q4_0 is the legacy 4-bit format without imatrix, kept mainly for cross-model on-device latency benchmarking. All other sub-8-bit variants use imatrix calibration.

Δ AVG below is measured against the FP32 GGUF baseline using our Normalized (0-100) score — the mean of 17 public VLM benchmarks (see Quality). Δ AVG (pts) is the absolute change; Δ AVG (rel %) normalizes by the baseline.

---

Quick Recommendation

---

Benchmarks

Flash keeps most of the full model's quality. All scores are computed in-house with a single VLMEvalKit harness so every configuration is scored identically, using each benchmark's official metric (POPE = F1, MMVet = partial credit, MM-IFEval = instruction-following accuracy, MME = Perception + Reasoning points, OCRBench = /1000; accuracy otherwise). LLM-as-judge scoring uses Qwen3-27B-FP8. Each column is a different GGUF quantization / calibration configuration.

Legend: -i = importance-matrix (imatrix) calibration. MME = P+R points, OCRBench = /1000, all others accuracy/F1. Normalized = mean across the 17 benchmarks scaled to 0-100 (MME /28, OCRBench /1000). Weights = on-disk LM GGUF only (excludes shared mmproj Q8_0, ~104 MB).

* ChartQA, TextVQA, DocVQA, InfoVQA, OCRBench, MathVista and MMVet use an LLM-as-judge Qwen3-27B-FP8, which evaluates free-form answers more reliably than strict string matching (e.g. "12%" vs "12 percent", paraphrases, units, formatting).

The trade-off vs the full VisionPsy-Nano-460M is concentrated in OCR/document-heavy and fine-grained perception tasks (OCRBench, ChartQA), where fewer visual tokens cost the most detail. For the full-quality model see the VisionPsy-Nano-460M-GGUF card.

---

Key Findings

• Q8_0 is effectively lossless: −0.06 pts (−0.10% relative) on Normalized score, at roughly half the size of BF16, no imatrix needed.
• Q4_K_M with imatrix is exceptional at 4-bit: only −0.53 pts (−0.92%), actually matching the 5-bit builds on Normalized score, making it the clear best size–quality trade-off for mobile and laptop deployment.
• imatrix beats legacy Q4_0 decisively: the imatrix 3-bit IQ3_M (−1.57 pts) and IQ3_XXS (−1.71 pts) both outperform the legacy 4-bit Q4_0 (−2.52 pts). Prefer an imatrix build unless you are specifically latency-benchmarking Q4_0.
• Compact quantization stays usable: even IQ3_M holds a Normalized score of 56.01, close to the FP32 Flash baseline (57.58), keeping Flash's speed advantage intact at a fraction of the footprint.

---

Usage

These are multimodal GGUF files: you need both the text model (*.gguf) and the vision projector (mmproj-*.gguf).

llama.cpp

# Download the recommended text weights (Q4_K_M with imatrix) + the vision projector
huggingface-cli download qvac/VisionPsy-Nano-460M-Flash-GGUFs \
    visionpsy-nano-460m-flash-q4_k_m-imat.gguf \
    mmproj-visionpsy-nano-460m-flash-q8.gguf \
    --local-dir .

First build the patched llama.cpp — see the build instructions in llama-cpp-inference, then run multimodal inference:

./llama-mtmd-cli \
    -m visionpsy-nano-460m-flash-q4_k_m-imat.gguf \
    --mmproj mmproj-visionpsy-nano-460m-flash-q8.gguf \
    --image your_image.jpg \
    -p "What is in this image?" \
    -n 128

# Or start an OpenAI-compatible server with a web UI
./llama-server \
    -m visionpsy-nano-460m-flash-q4_k_m-imat.gguf \
    --mmproj mmproj-visionpsy-nano-460m-flash-q8.gguf

---

Intended use

VisionPsy-Nano-460M-Flash-GGUFs is designed for latency- and memory-constrained, on-device multimodal applications where responsiveness matters most: live camera Q&A, quick scene/document understanding, and lightweight visual instruction following on phones and other edge hardware. If you can afford more compute and want maximum quality, use VisionPsy-Nano-460M-GGUFs instead. Because of its small size, we recommend fine-tuning on your specific domain to maximize quality.

Limitations

• Flash trades a little accuracy for speed: it scores a few points below the full VisionPsy-Nano-460M, with the largest gaps on OCR/document-heavy and fine-grained perception tasks.
• Single-image by design: the model is trained and optimized for one image per query; multi-image prompts are outside its intended use.
• As a compact model, it may occasionally hallucinate or miscount and is best suited to focused tasks rather than very dense documents or long multi-step math, where larger models have an edge.
• Quantization artifacts: lower bit counts can subtly degrade outputs in ways aggregate benchmarks do not fully capture. We recommend Q4_K_M or higher for production and prefer the imatrix builds over the legacy Q4_0.
• Primarily English; other languages are not officially supported yet.
• Not intended for safety-critical or high-stakes automated decisions.
• Efficiency numbers are measured with 4-bit GGUF builds on specific phones; absolute latency, memory, and throughput will vary with hardware, runtime, and quantization. Benchmark scores use a fixed in-house harness with an LLM judge (Qwen3-27B-FP8) and may differ from other reported setups.

Acknowledgements

Built on the excellent open-source work of nanoVLM, SmolLM2, SigLIP2, and llama.cpp.

Citation

@misc{visionpsynano2026,
title = {VisionPsy-Nano-460M: A Compact Vision-Language Model for On-Device Inference},
author = {Tether AI Research},
year = {2026},
note = {Hugging Face model card}
}

Copyright

We will take appropriate actions in response to notices of copyright infringement. If you believe your work has been used or copied in a manner that infringes upon your intellectual property rights, please email [email protected] identifying and describing both the copyrighted work and alleged infringing content.

Licensing

This model, which was finetuned as described in the blog post, is licensed by Tether Data, S.A. de C.V. under the Apache 2.0 license. As described in the blog post, this model is a version of the NanoVLM-460M-8k pre-trained model (https://huggingface.co/lusxvr/nanoVLM-460M-8k), which is made available under the MIT license.

The FineVision dataset (https://huggingface.co/datasets/HuggingFaceM4/FineVision) is made available under the CC-BY-4.0 (Creative Commons - Attribution 4.0) license. FineVision is an aggregation of a number of public sources unified into a single corpus. Individual subsets within the collection may inherit specific underlying terms from their original creators. As described in the blog post, a subset of the FineVision dataset was used as a part of finetuning the model.

The NVIDIA Nemotron-Image-Training-v3 dataset (https://huggingface.co/datasets/nvidia/Nemotron-Image-Training-v3) is made available under the CC-BY-4.0 (Creative Commons - Attribution 4.0). The mPLUG TinyChartData dataset (https://huggingface.co/datasets/mPLUG/TinyChartData) is made available under the Apache 2.0 license. The TabMWP dataset (https://promptpg.github.io/) is made available under the CC BY-NC-SA 4.0 (Creative-Commons-Attribution-NonCommercial-ShareAlike 4.0). The PopVQA dataset (https://huggingface.co/datasets/idoco/PopVQA) is made available under the MIT license. The InfoSeek dataset (https://github.com/open-vision-language/infoseek) is made available under the Apache 2.0 license. The MMKU-Bench dataset (https://huggingface.co/datasets/baochenfu/MMKU-Bench) is made available under the Apache 2.0 license. The VisionFoundry-10K dataset (https://huggingface.co/datasets/zlab-princeton/VisionFoundry-10K) is made available under the Apache 2.0 license. The PKU-SafeRLHF-V dataset (https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF-V) is made available under the CC-BY-NC 4.0 (Attribution-NonCommercial 4.0 International). As described in the blog post, the NVIDIA Nemotron-Image-Training-v3, mPLUG TinyChartData, TabMWP, PopVQA, InfoSeek, MMKU-Bench, VisionFoundry-10K and PKU-SafeRLHF-V datasets were used as a part of finetuning the model.