VisionPsy-Nano-460M-GGUFs

VisionPsy-Nano-460M-GGUFs

VisionPsy-Nano-460M-GGUFs provides GGUF weights of VisionPsy-Nano-460M for fast, fully on-device multimodal inference with our patched llama.cpp fork (not stock upstream). Requires the patched llama.cpp. VisionPsy-Nano-460M is a compact (~460M parameter) vision-language model that compares favourably with other models in its ~0.5B weight class on 16 of 17 public benchmarks; these GGUF builds bring that quality to phones, tablets, and edge hardware.

We release some GGUF variants, ranging from unquantized FP32 and BF16 references to lossless 8-bit, high-quality 5-bit, and ultra-compact 3-bit formats. This range supports deployment across diverse hardware settings, from high-end workstations to resource-constrained on-device environments. All sub-8-bit variants (except the legacy Q4_0) use importance-matrix (imatrix) calibration, which consistently reduces quality degradation.

Each GGUF is a vision-language model: alongside the text weights (*.gguf) you also need the multimodal projector (mmproj-*.gguf, the SigLIP2 vision encoder) to run image inputs. See Usage.

^* References to Tether AI Research are references to Tether Data, S.A. de C.V.

---

Available Files

All published GGUF files are generated with llama.cpp. Quantization labels refer exclusively to the language-model weights, while the vision encoder and multimodal projector are fixed to Q8_0 for all variants. FP32 and BF16 are unquantized language-model references evaluated within the same inference harness. Q8_0 does not use importance-matrix (imatrix) calibration, as it provides no measurable benefit at 8-bit precision. Q4_0 is the legacy 4-bit format without imatrix calibration and is retained mainly for cross-model on-device latency comparisons. All remaining sub-8-bit language-model variants use imatrix calibration.

Δ AVG below is measured against the FP32 GGUF baseline using our Normalized (0-100) score, the mean of 17 public VLM benchmarks (see Benchmarks). Δ AVG (pts) is the change in points; Δ AVG (rel %) normalizes by the baseline.

---

Quick Recommendation

---

Benchmarks

All scores are computed in-house with a single VLMEvalKit harness so every configuration is scored identically, using each benchmark's official metric (POPE = F1, MMVet = partial credit, MM-IFEval = instruction-following accuracy, MME = Perception + Reasoning points, OCRBench = Final Score /1000; accuracy otherwise). LLM-as-judge scoring uses Qwen3-27B-FP8. Each column is a different GGUF quantization / calibration configuration.

Legend: -i = importance-matrix (imatrix) calibration. MME = P+R points, OCRBench = /1000, all others accuracy/F1. Normalized = mean across the 17 benchmarks scaled to 0-100 (MME /28, OCRBench /1000). Weights = on-disk LM GGUF only (excludes shared mmproj Q8_0, ~104 MB).

* ChartQA, TextVQA, DocVQA, InfoVQA, OCRBench, MathVista, and MMVet use an LLM-as-judge Qwen3-27B-FP8, which evaluates free-form answers more reliably than strict string matching (e.g. "12%" vs "12 percent", paraphrases, units, formatting).

---

Key Findings

• Q8_0 is effectively lossless: no difference on the Normalized score, at roughly half the size of BF16, with no imatrix needed.
• Q5_K_M is the recommended high-quality option: only −0.12 pts (−0.19%), and it is also better than Q5_K_M with imatrix (−0.59 pts).
• Q4_K_M with imatrix is the sweet spot: −0.24 pts (−0.39%) for a substantial size reduction, the best size/quality choice for mobile/laptop.
• imatrix beats legacy Q4_0 decisively: the imatrix 3-bit IQ3_M (−1.36 pts) and even IQ3_XXS (−1.70 pts) both outperform the legacy 4-bit Q4_0 (−2.85 pts). If you are not latency-benchmarking, there is no reason to prefer Q4_0 over an imatrix build.
• Compact 3-bit stays strong: IQ3_M holds a Normalized score of 60.39, making it a viable option for the tightest memory budgets.

---

Usage

These are multimodal GGUF files: you need both the text model (*.gguf) and the vision projector (mmproj-*.gguf).

llama.cpp

# Download the recommended text weights (Q4_K_M with imatrix) + the vision projector
huggingface-cli download qvac/VisionPsy-Nano-460M-GGUFs \
    visionpsy-nano-460m-q4_k_m-imat.gguf \
    mmproj-visionpsy-nano-460m-q8.gguf \
    --local-dir .

First build the patched llama.cpp — see the build instructions in llama-cpp-inference, then run multimodal inference:

# Run multimodal inference (image + prompt)
./llama-mtmd-cli \
    -m visionpsy-nano-460m-q4_k_m-imat.gguf \
    --mmproj mmproj-visionpsy-nano-460m-q8.gguf \
    --image your_image.jpg \
    -p "What is in this image?" \
    -n 128

# Or start an OpenAI-compatible server with a web UI
./llama-server \
    -m visionpsy-nano-460m-q4_k_m-imat.gguf \
    --mmproj mmproj-visionpsy-nano-460m-q8.gguf

---

Intended use

VisionPsy-Nano-460M-GGUFs targets latency- and memory-constrained, on-device multimodal applications: visual question answering, document/chart/diagram understanding, scene-text reading, and lightweight visual instruction following on phones and other edge hardware. Because of its small size, we recommend fine-tuning on your specific domain to maximize quality.

Limitations

• Single-image by design: the model is trained and optimized for one image per query; multi-image prompts are outside its intended use.
• As a compact model, it may occasionally hallucinate or miscount and is best suited to focused tasks rather than very dense documents or long multi-step math, where larger models have an edge.
• Quantization artifacts: lower bit counts can subtly degrade outputs in ways aggregate benchmarks do not fully capture. We recommend Q4_K_M or higher for production and prefer the imatrix builds over the legacy Q4_0.
• Primarily English; other languages are not officially supported yet.
• Not intended for safety-critical or high-stakes automated decisions.
• Benchmark scores are produced with a fixed in-house harness and an LLM judge (Qwen3-27B-FP8); absolute numbers may differ from other reported setups.

Acknowledgements

Built on the excellent open-source work of nanoVLM, SmolLM2, SigLIP2, and llama.cpp.

Citation

@misc{visionpsynano2026,
title = {VisionPsy-Nano-460M: A Compact Vision-Language Model for On-Device Inference},
author = {Tether AI Research},
year = {2026},
note = {Hugging Face model card}
}

Copyright

We will take appropriate actions in response to notices of copyright infringement. If you believe your work has been used or copied in a manner that infringes upon your intellectual property rights, please email [email protected] identifying and describing both the copyrighted work and alleged infringing content.

Licensing

This model, which was finetuned as described in the blog post, is licensed by Tether Data, S.A. de C.V. under the Apache 2.0 license. As described in the blog post, this model is a version of the NanoVLM-460M-8k pre-trained model (https://huggingface.co/lusxvr/nanoVLM-460M-8k), which is made available under the MIT license.

The FineVision dataset (https://huggingface.co/datasets/HuggingFaceM4/FineVision) is made available under the CC-BY-4.0 (Creative Commons - Attribution 4.0) license. FineVision is an aggregation of a number of public sources unified into a single corpus. Individual subsets within the collection may inherit specific underlying terms from their original creators. As described in the blog post, a subset of the FineVision dataset was used as a part of finetuning the model.

The NVIDIA Nemotron-Image-Training-v3 dataset (https://huggingface.co/datasets/nvidia/Nemotron-Image-Training-v3) is made available under the CC-BY-4.0 (Creative Commons - Attribution 4.0). The mPLUG TinyChartData dataset (https://huggingface.co/datasets/mPLUG/TinyChartData) is made available under the Apache 2.0 license. The TabMWP dataset (https://promptpg.github.io/) is made available under the CC BY-NC-SA 4.0 (Creative-Commons-Attribution-NonCommercial-ShareAlike 4.0). The PopVQA dataset (https://huggingface.co/datasets/idoco/PopVQA) is made available under the MIT license. The InfoSeek dataset (https://github.com/open-vision-language/infoseek) is made available under the Apache 2.0 license. The MMKU-Bench dataset (https://huggingface.co/datasets/baochenfu/MMKU-Bench) is made available under the Apache 2.0 license. The VisionFoundry-10K dataset (https://huggingface.co/datasets/zlab-princeton/VisionFoundry-10K) is made available under the Apache 2.0 license. The PKU-SafeRLHF-V dataset (https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF-V) is made available under the CC-BY-NC 4.0 (Attribution-NonCommercial 4.0 International). As described in the blog post, the NVIDIA Nemotron-Image-Training-v3, mPLUG TinyChartData, TabMWP, PopVQA, InfoSeek, MMKU-Bench, VisionFoundry-10K and PKU-SafeRLHF-V datasets were used as a part of finetuning the model.