Downloads · 30 days
512
41% of all-time downloads
gemma-4-12B-it-heretic-volta-gguf
megabytes/gemma-4-12B-it-heretic-volta-gguf
gemma-4-12B-it-heretic-volta-gguf is a image-text-to-text model from megabytes. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This repository contains the complete Gemma 4 12B IT QAT Heretic suite specifically optimized for NVIDIA Volta (Compute Capability 7.0 / Tesla V100 / Titan V) GPUs.
All-time downloads
1.2K
Public
Repo size
15.2 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf7.6 GB · 100%
At a glance
- Task
- Image-Text-to-Text
- License
- apache-2.0
- Access
- Public
- Created
- Sep 19, 2026
- Updated
- Sep 19, 2026
- SHA
- c53e71fc
Try a prompt
Base models
Model card
From the Hugging Face model README
megabytes/gemma-4-12B-it-heretic-volta-gguf
This repository contains the complete Gemma 4 12B IT QAT Heretic suite specifically optimized for NVIDIA Volta (Compute Capability 7.0 / Tesla V100 / Titan V) GPUs.
Included Models
| File | Size | Format | Description |
|---|---|---|---|
gemma-4-12b-it-qat-heretic-volta-Q4_0.gguf | 6.8 GB | Q4_0 + Q8_0 | Main LLM with Q8_0 token embeddings (avoids Q6_K stalls on Volta) and canonical chat template |
mmproj-gemma-4-12b-it-qat-heretic-volta-F16.gguf | 117 MB | F16 | Multimodal vision projector mapped directly to Volta FP16 Tensor Cores |
mtp-gemma-4-12b-it-qat-heretic-volta-Q4_0.gguf | 242 MB | Q4_0 | Multi-Token Prediction (MTP) speculative decoding assistant model |
Optimizations for Volta (sm_70)
- Avoidance of K-Quants (
Q6_K→Q8_0): Eliminates ALU unpacking bottlenecks on Volta CUDA cores for the 262k-token vocabulary. - Native FP16 Vision Projector (
F16): Runs vision embeddings directly on Volta's FP16 Tensor Cores without integer dequantization overhead. - Speculative Decoding Assistant (
Q4_0): Compressed to 242 MB to maximize speculative drafting throughput. - Canonical Delimiters & Chat Template: Fully aligned with Gemma 4 turn delimiters (
<|turn>,<turn|>) and Jinja chat templating.
Benchmark Performance (Tesla V100-PCIE-12GB)
| Workload | Throughput | Notes |
|---|---|---|
Synthetic Prompt (pp512) | 270.2 t/s | Raw base model throughput |
Synthetic Generation (tg128) | 50.5 t/s | Raw base model throughput |
| Text Generation (with MTP drafter) | 59.5 t/s | --spec-draft-n-max 2 --poll 100 |
| Multimodal Vision Ingestion | 94.4 t/s | Native F16 projector |
Recommended llama.cpp / llama-server Command
./llama-server \
-m ./gemma-4-12b-it-qat-heretic-volta-Q4_0.gguf \
--mmproj ./mmproj-gemma-4-12b-it-qat-heretic-volta-F16.gguf \
--model-draft ./mtp-gemma-4-12b-it-qat-heretic-volta-Q4_0.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 2 \
--poll 100 \
-dev CUDA0 \
-ngl all \
-devd CUDA0 \
-ngld all \
--mmproj-offload \
--spec-draft-backend-sampling \
-fa on \
--jinja
- Task
- Image-Text-to-Text
- License
- apache-2.0
- Created
- Sep 19, 2026
- Updated
- Sep 19, 2026