Mistral-Heretica-12B-GGUF

Mistral-Heretica-12B-GGUF

GGUF quantizations of mrcuddle/Mistral-Heretica-12B.

Original model: a Task Arithmetic merge of Mistral-Nemo-Instruct-2407, Lumimaid-v0.2-12B, and absolute-heresy — tuned for uncensored roleplay and creative writing. 12B params, Mistral architecture.

Quantized with llama.cpp (release b9890). Run these in LM Studio, Ollama, KoboldCpp, SillyTavern, or llama.cpp directly.

Download

F16 (23 GB) is the unquantized conversion — only needed if you want to re-quantize yourself.

Which file should I choose?

Pick the largest quant that fits in your RAM/VRAM with room to spare (leave ~1–2 GB for context).

• 8 GB RAM/VRAM → Q4_K_M
• 12 GB → Q5_K_M or Q6_K
• 16 GB+ → Q6_K or Q8_0

For roleplay/creative use, Q4_K_M is plenty — quality loss over Q8 is negligible in practice.

Prompt format

Mistral instruct format (primary):

<s>[INST] {prompt} [/INST]

Most front-ends (SillyTavern, LM Studio) apply this automatically when you select a Mistral / Mistral-Nemo template. The Lumimaid component was also trained with ChatML, so ChatML works too — but Mistral format is the safe default.

Recommended settings

Mistral-Nemo (this model's base) is temperature-sensitive — high temperature makes it incoherent. Keep it low:

Start at temp 0.6 / min-p 0.05 and adjust from there.

Run it

llama.cpp:

llama-cli -m Mistral-Heretica-12B-GGUF-Q4_K_M.gguf --jinja -p "Write a short noir opening."

Download one file with the HF CLI (skip the rest):

hf download AnkitAI/Mistral-Heretica-12B-GGUF \
  Mistral-Heretica-12B-GGUF-Q4_K_M.gguf --local-dir .

Ollama:

ollama run hf.co/AnkitAI/Mistral-Heretica-12B-GGUF:Q4_K_M

Verified

Q4_K_M smoke-tested: loads correctly, follows instructions (valid JSON output), and produces coherent creative prose. ~5–6 tok/s generation on Apple Silicon (M-series).

Support the Project

If this model is useful in your work, you can support independent research:

<p align="left"> <a href="https://www.buymeacoffee.com/AnkitAI" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee" height="60" width="217" /></a> </p>

License

The base model declares no explicit license. This merge inherits terms from its source models — notably Lumimaid-v0.2-12B, which is CC-BY-NC-4.0 (non-commercial). Treat this model as non-commercial unless you verify otherwise with the original authors. These are quantizations only; all model rights belong to the original creators.

Credits

• Original model: mrcuddle
• Merge components: NeverSleep, MuXodious, Mistral AI
• Quantization tooling: llama.cpp

More models: ankitaglawe.com