Downloads · 30 days
299
13% of all-time downloads
Qwen3.5-2B-heretic-GGUF
Abiray/Qwen3.5-2B-heretic-GGUF
Qwen3.5-2B-heretic-GGUF is a image-text-to-text model from Abiray. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product.
This repository contains GGUF quantizations for jordanwoodson/Qwen3.5-2B-heretic, alongside the mmproj-F16.gguf vision projector to enable multimodal (image-to-text) inference.
All-time downloads
2.3K
Public
Repo size
6.9 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.gguf6.9 GB · 100%
At a glance
- Task
- Image-Text-to-Text
- Access
- Public
- Created
- Mar 9, 2026
- Updated
- Mar 9, 2026
- SHA
- 2393e0e7
Try a prompt
Base models
Model card
From the Hugging Face model README
Qwen3.5-2B-heretic-GGUF
This repository contains GGUF quantizations for jordanwoodson/Qwen3.5-2B-heretic, alongside the mmproj-F16.gguf vision projector to enable multimodal (image-to-text) inference.
This model is good for edge devices and better performance.
Stop Sequence
Please ensure you update the stop sequence in your inference client to:
<|im_end|>
Available Files
| Filename | Format | Description |
|---|---|---|
Qwen3.5-2B-heretic-Q4_K_M.gguf | Q4_K_M | Ideal sweet spot. Excellent balance of low memory footprint and high quality. |
Qwen3.5-2B-heretic-Q5_K_M.gguf | Q5_K_M | Slightly larger than Q4, offering a marginal quality increase. |
Qwen3.5-2B-heretic-Q6_K.gguf | Q6_K | Very high quality, nearly indistinguishable from unquantized. |
Qwen3.5-2B-heretic-Q8_0.gguf | Q8_0 | Practically lossless fidelity, largest quantized size. |
mmproj-F16.gguf | F16 | The vision projector required for any image analysis tasks. |
Usage with llama.cpp
You can run this model locally using the llama.cpp command-line interface.
Standard Text Inference
To run a standard text generation prompt, point llama-cli to your downloaded GGUF file:
./llama-cli -m Qwen3.5-2B-heretic-Q4_K_M.gguf -p "Write a quick Python script." -n 512
- Task
- Image-Text-to-Text
- Created
- Mar 9, 2026
- Updated
- Mar 9, 2026