Downloads · 30 days
165
78% of all-time downloads
sjoe1244/gemma-4-31B-it-heretic-exl3-4.00bpw-h6
gemma-4-31B-it-heretic-exl3-4.00bpw-h6 is a image-text-to-text model from sjoe1244. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
Unofficial EXL3 export of coder3101/gemma-4-31B-it-heretic for local ExLlamaV3 / TabbyAPI serving.
Downloads · 30 days
165
78% of all-time downloads
All-time downloads
211
Public
Parameters
9.8B
19.7 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors19.7 GB · 100%
How the weights are stored.
I167.9B · 80%
From the Hugging Face model README
Unofficial EXL3 export of
coder3101/gemma-4-31B-it-heretic
for local ExLlamaV3 / TabbyAPI serving.
This is a 4.00 bpw / h6 convert with the Gemma 4 vision tensors left in BF16 inside the same checkpoint. It is meant for 24 GB cards that can hold ~18.6 GB weights plus Q8 KV.
| File | Notes |
|---|---|
model-00001-of-00003.safetensors … 00003 | EXL3 weights + BF16 vision |
quantization_config.json | EXL3 4.00 bpw, head_bits 6, codebook mcg |
preprocessor_config.json / processor_config.json | Gemma 4 vision processor |
tokenizer.json | tokenizer; no chat_template in tokenizer_config.json |
gemma4.jinja | Gemma 4 chat template to point TabbyAPI at |
Do not use this folder as a Transformers / bitsandbytes load. Load it with
ExLlamaV3 or TabbyAPI's exllamav3 loader.
From quantization_config.json:
0.0.29Converted locally on an NVIDIA RTX 4090 (August 2026).
Worked here in TabbyAPI:
cache_mode: Q8cache_size: 8192vision: true (or omit; vision tensors are in the shards either way)chunk_size: 2048prompt_template at gemma4.jinja if the tokenizer has no chat templateRough VRAM with vision + Q8 8K KV: ~18.6 GB.
Gemma 4 terms apply to the original Google weights. Heretic / abliteration
credit belongs to coder3101.
This repo is only the EXL3 export.