Goetia-26B-A4B-v1.3-Heretic-ARA-FP8-Dynamic

Goetia-26B-A4B-v1.3-Heretic-ARA-FP8-Dynamic

FP8 Dynamic (W8A8, FP8_DYNAMIC) quantization of Naphula/Goetia-26B-A4B-v1.3-Absolute-Heretic-ARA.

Source

• Original model: Naphula/Goetia-26B-A4B-v1.3-Absolute-Heretic-ARA (Gemma-4-26B-A4B MoE, BF16), a roleplay finetune with abliteration-style refusal removal ("Absolute Heretic ARA").
• License: apache-2.0, inherited from the original model.

How it was made

llm-compressor, data-free one-shot, with the exact same recipe as the production quant cloud19/G4-MeroMero-26B-FP8-Dynamic-Uncensored:

default_stage:
  default_modifiers:
    QuantizationModifier:
      targets: [Linear]
      ignore: [lm_head, 're:.*embed.*', 're:.*router', 're:.*vision_tower.*']
      scheme: FP8_DYNAMIC
      bypass_divisibility_checks: false

Weights are FP8 per-channel (static), activations FP8 per-token (dynamic). Kept in BF16: embeddings, lm_head, MoE routers and the vision tower.

Weights size: 25.30 GiB (BF16 original ~48.1 GiB). Fits a single 96 GB GPU with ample room for KV cache.

Serving with vLLM

Intended for text-only serving.

vllm serve cloud19/Goetia-26B-A4B-v1.3-Heretic-ARA-FP8-Dynamic \
  --served-model-name goetia-ara \
  --quantization compressed-tensors \
  --kv-cache-dtype fp8 \
  --max-model-len 16384 \
  --max-num-seqs 384 \
  --limit-mm-per-prompt '{"image": 0}' \
  --no-enable-chunked-prefill