Downloads · 30 days
83
28% of all-time downloads
mazenDDr/receipt-vlm-qwen2.5-vl-3b-GGUF
receipt-vlm-qwen2.5-vl-3b-GGUF is a image-text-to-text model from mazenDDr. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
The fine-tuned model for llama.cpp, as Q4KM text weights plus the vision projector.
Downloads · 30 days
83
28% of all-time downloads
All-time downloads
299
Public
Repo size
4.6 GB
Likes
0
Public
Click a slice to open those files.
.gguf4.6 GB · 100%
From the Hugging Face model README
The fine-tuned model for llama.cpp, as Q4_K_M text weights plus the vision projector.
| File | Size | What it is |
|---|---|---|
model-Q4_K_M.gguf | 1.79 GiB | the language model, 4-bit |
mmproj-f32.gguf | 2.49 GiB | the vision projector — required |
| Field F1 | Latency p50 | Output tok/s | |
|---|---|---|---|
| GGUF bf16 (reference) | 0.846 | 2.57 s | 41.4 |
| Q4_K_M | 0.832 | 1.51 s | 75.4 |
Paired against that bf16 reference on the same 100 receipts: −0.014 [−0.038, +0.009]. Not real.
llama-server -m model-Q4_K_M.gguf --mmproj mmproj-f32.gguf \
--image-max-tokens 1024 -c 4096 -ngl 99 --temp 0 --top-k 1 --repeat-penalty 1.0
The projector must be f32. An f16 projector overflows into NaN on particular receipts, and the model
then emits ! until it hits the generation cap — content-dependent, deterministic, and completely
silent: the server logs normal throughput throughout. Measured on ten receipts, an f16 projector flooded
six of them and scored field F1 0.540 with 60% truncated; the f32 projector scored 0.899 with nothing
truncated. That is why only the f32 file is published here.
It also cannot be quantized, so it is a fixed 2.49 GiB on every row. The total here (4.28 GiB) is larger than the AWQ checkpoint at 3.31 GiB — lower bit width did not produce a smaller deployment.
Q3_K_M and Q2_K are not published. Without an importance matrix they collapse (field F1 0.100 and 0.000 — Q2_K emits a median of three tokens). With one they recover on dev, but Q2_K+imatrix is then genuinely worse on test (−0.054 [−0.092, −0.020]). Q4_K_M is the row worth shipping.
QLoRA on one RTX 5060 Ti (16 GB): the base model in 4-bit NF4 with the vision tower kept in bf16, LoRA rank 16 / alpha 32 on the language-model layers only, learning rate 4e-4, 2 epochs, loss on the answer tokens alone. 1.07 h, 7.6 GB peak VRAM, 29.9 M trainable parameters.
Training data is CORD v2 (CC BY 4.0), split 773 / 99 / 100 after dropping 28 images that appeared in more than one split under a 256-bit perceptual hash. Every choice was made on the dev split; the test split was scored once.
. and , interchangeably as thousands separators. The model
learned those conventions; on receipts from elsewhere, formatting may not match your expectations.Receipts from CORD v2 (Park et al., CORD: A Consolidated Receipt Dataset for Post-OCR Parsing, 2019), CC BY 4.0. Base model Qwen/Qwen2.5-VL-3B-Instruct, Apache-2.0.