gemma-4-E4B-it-heretic-GGUF

GGUF quantizations of igorls/gemma-4-E4B-it-heretic

Disclaimer: This model has had its safety alignment removed through abliteration. It will comply with requests that the original model would refuse. This model is provided for research and educational purposes. Users are solely responsible for how they use this model and any content it generates. The creators assume no liability for misuse.

Abliterated (decensored) version of google/gemma-4-E4B-it, created using Heretic v1.2.0 with Arbitrary-Rank Ablation (ARA) and row-norm preservation.

Available quantizations

Performance

The abliterated model slightly outperforms the base on PIQA (+0.0033) while reducing refusals from 99/100 to 3/100.

Vision Support

Note: Vision is not supported via GGUF/Ollama for Gemma 4 custom models (as of Ollama v0.20.0). Ollama's native gemma4 renderer does not expose vision to the llama.cpp backend for custom GGUFs.

For vision tasks, use the safetensors version via Transformers or Unsloth: igorls/gemma-4-E4B-it-heretic

The mmproj-F16.gguf is included for future compatibility when Ollama adds Gemma 4 vision support for custom GGUFs.

Usage with Ollama

# Download the GGUF (e.g. Q8_0)
# Create a Modelfile:
cat > Modelfile << 'EOF'
FROM ./gemma-4-E4B-it-heretic-Q8_0.gguf
SYSTEM "You are a helpful assistant."
PARAMETER temperature 0.7
PARAMETER top_p 0.95
PARAMETER top_k 64
EOF

# Import and run
ollama create gemma4-e4b-heretic -f Modelfile
ollama run gemma4-e4b-heretic

Usage with llama.cpp

llama-cli -m gemma-4-E4B-it-heretic-Q4_K_M.gguf -p "Hello, how are you?" -n 256

Abliteration parameters (ARA)