Downloads · 30 days
11
5% of all-time downloads
Nabhos/Gemma-4-31B-it-abliterated-heretic-text-only
Gemma-4-31B-it-abliterated-heretic-text-only is a image-text-to-text model from Nabhos. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
AWQ W4A16 (groupsize 128, symmetric) quantization of trohrbaugh/gemma-4-31b-it-heretic-ara — a Heretic-ARA abliterated derivative of google/gemma-4-31b-it.
Downloads · 30 days
11
5% of all-time downloads
All-time downloads
216
Public
Parameters
30.9B
18 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors17.9 GB · 100%
How the weights are stored.
I3229.3B · 95%
From the Hugging Face model README
AWQ W4A16 (group_size 128, symmetric) quantization of trohrbaugh/gemma-4-31b-it-heretic-ara — a Heretic-ARA abliterated derivative of google/gemma-4-31b-it.
⚠️ Decensored model. Safety guardrails have been deliberately removed. Research and experimentation only. See full disclaimer below.
| Parameter | Value |
|---|---|
| Method | AWQ (Activation-aware Weight Quantization) |
| Scheme | W4A16 (symmetric) |
| Weight Bits | 4 |
| Activation Bits | 16 |
| Group Size | 128 |
| Format | compressed-tensors |
| Calibration Dataset | HuggingFaceH4/ultrachat_200k |
| Calibration Samples | 256 |
| Max Sequence Length | 2048 |
| Vision Tower | Unquantized (full precision) |
| LM Head | Unquantized (full precision) |
| Compatible Inference Engine | vLLM (vllm/vllm-openai:gemma4) |
v_proj → o_proj on this checkpoint across many
of Gemma 4's decoder blocks. Per-channel smoothing is skipped for that path,
matching the standard AWQ-for-GQA pattern.recipe.yaml in this repo for
reproducibility. It records the exact AWQ mappings, ignore patterns, and
scheme parameters applied.Recommended inference with vLLM (Gemma 4 requires the vllm/vllm-openai:gemma4 image):
vllm serve alonsoko/gemma-4-31b-it-abliterated-heretic-AWQ-W4A16 \
--trust-remote-code \
--tensor-parallel-size 1 \
--max-model-len 32768
Recommended sampling (per upstream Gemma 4 guidance):
temperature=1.0, top_p=0.95, top_k=64
To enable thinking mode, include the <|think|> token at the start of the system prompt.
Approximate VRAM for inference at this quantization (W4A16-G128):
This model is intended for research, experimentation, and testing purposes only.
This model is shared purely for academic and technical exploration of model internals.
Performed with Heretic v1.2.0+custom using the Arbitrary-Rank Ablation (ARA) method.
| Parameter | Value |
|---|---|
| start_layer_index | 2 |
| end_layer_index | 60 |
| preserve_good_behavior_weight | 0.9920 |
| steer_bad_behavior_weight | 0.0001 |
| overcorrect_relative_weight | 0.4709 |
| neighbor_count | 10 |
| Metric | This model | Original google/gemma-4-31b-it |
|---|---|---|
| KL divergence | 0.0120 | 0 (by definition) |
| Refusals | 5/100 | 98/100 |
Measured on the unquantized heretic-ara base; AWQ is expected to preserve these closely but has not been separately benchmarked.
Original model: google/gemma-4-31b-it
Gemma 4 31B is a dense multimodal model (text + image input, text output) with a 256K context window, native thinking-mode support, function calling, and strong performance on reasoning, coding, and vision benchmarks. See the base model card for architectural details, benchmark results, training data, and Google's responsible-use guidance.
<div align="center"> <img src="https://ai.google.dev/gemma/images/gemma4_banner.png" alt="Gemma 4"> </div>