Gemmable-4-12B-MTP-GGUF

<p align="center"> <img src="assets/gemmable_4_12b.png" alt="Gemmable 4 12b" width="720"> </p>

Gemmable 4 12B

Gemmable 4 12B is a GGUF export of Gemma 4 12B fine-tuned on Fable-5 style reasoning and assistant traces.

<a href="https://x.com/MiaAI_lab" target="_blank"> <img src="https://img.shields.io/badge/Follow%20me%20on%20X-000000?style=for-the-badge&logo=x&logoColor=white" alt="Follow Mia on X" /> </a> <a href='https://ko-fi.com/Z8Z3SPLOD' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi6.png?v=6' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a>

Highlights

• Base model: google/gemma-4-12B
• Format: GGUF
• Training style: Fable-5 style reasoning and assistant traces
• Distribution: fp16 GGUF plus matching assistant GGUFs for each quant
• Intended use: local inference, coding, reasoning, and assistant workflows

How to use

llama.cpp

Standard load:

llama-server -m "gemmable-4-12b-fp16.gguf"

Speculative / draft-MTP load:

llama-server -m "gemmable-4-12b-Q4_K_M.gguf" \
  --spec-draft-model "gemmable-4-12b-Q4_K_M-mtp.gguf" \
  --spec-type draft-mtp \
  --spec-draft-n-max 4

Use the matching fp16 or quantized main file with its -mtp companion.

LM Studio

1. Search this repo, download target + mtp file.
2. Load target.
3. Load settings → Speculative Decoding → select mtp file file.

(Requires LM Studio with am17an's PR merged or custom llama.cpp runtime. As of 2026-05, mainline LM Studio runtime doesn't yet have draft-mtp for Gemma-4 — track upstream merge.)

GGUF / local inference notes

• gemmable-4-12b-fp16.gguf is the standard fp16 main model.
• gemmable-4-12b-fp16-mtp.gguf is the matching fp16 assistant / draft file.
• Quantized pairs follow the same pattern, for example gemmable-4-12b-Q4_K_M.gguf and gemmable-4-12b-Q4_K_M-mtp.gguf.
• Keep the paired files in the same Hugging Face repository if you upload them.

Limitations

• Output quality depends on the prompt and runtime settings.
• MTP / speculative support depends on the client application.
• This is not a raw base checkpoint; it is a fine-tuned export.

Naming

Gemmable = Gemma + Fable-style tuning.