Downloads · 30 days
9
1% of all-time downloads
Infinity-Parser2-Flash-AWQ-W4A16
spectator2026/Infinity-Parser2-Flash-AWQ-W4A16
Infinity-Parser2-Flash-AWQ-W4A16 is a image-text-to-text model from spectator2026. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
An int4 (W4A16) AWQ quantization of infly/Infinity-Parser2-Flash, made to run on NVIDIA A100 / sm80, where the original FP8 path is unsupported. ~4.2 GB bf16 → ~2.9 GB.
All-time downloads
733
Public
Parameters
2.2B
3.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.1 GB · 99%
Parameter types
How the weights are stored.
BF161.3B · 59%
Try a prompt
Base models
Model card
From the Hugging Face model README
Infinity-Parser2-Flash — AWQ W4A16 (int4)
An int4 (W4A16) AWQ quantization of infly/Infinity-Parser2-Flash, made to run on NVIDIA A100 / sm80, where the original FP8 path is unsupported. ~4.2 GB bf16 → ~2.9 GB.
Method
AWQ via llm-compressor, routed experts only (W4A16). Attention, GDN/linear-attention, shared expert, vision tower, and lm_head are kept in bf16. Calibrated on a few hundred diverse document + general-vision samples.
Serving note (important)
vLLM fuses some bf16 layers before consulting the ignore list, which can otherwise yield all-! output. Fix: the saved config.json quantization_config.ignore uses broad regexes matching the fused names. Already applied here.
Quality (VLMEvalKit, AI-judged reproduction)
| Benchmark | Published bf16 | This int4 |
|---|---|---|
| MMStar | 57.1 | 54.8 |
| OCRBench | 81.6 | 85.0 |
| DocVQA (val) | 93.2 | 93.5 |
Near-lossless on the cleanly-comparable axes. (MMBench omitted to avoid a circular-vs-vanilla scoring mismatch.)
Usage (vLLM)
vllm serve spectator2026/Infinity-Parser2-Flash-AWQ-W4A16 --dtype bfloat16 --trust-remote-code --reasoning-parser qwen3
Pass chat_template_kwargs={"enable_thinking": false} in requests, or answers land in the reasoning channel.
Quantized by @spectator2026. Original model © infly, Apache-2.0.
- Task
- Image-Text-to-Text
- Type
- qwen3_5
- License
- apache-2.0
- Created
- May 30, 2026
- Updated
- May 30, 2026