Downloads · 30 days
116
9% of all-time downloads
jackasda211233/Gemma-4-E4B-Deckard-AttnQ8-IQ4_NL-storyMaker
Gemma-4-E4B-Deckard-AttnQ8-IQ4_NL-storyMaker is a machine learning model from jackasda211233. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for llama.cpp. The card lists the license as apache-2.0.
This is a normal llama.cpp GGUF quant of DavidAU's Deckard Gemma model:
Downloads · 30 days
116
9% of all-time downloads
All-time downloads
1.3K
Public
Repo size
5.2 GB
Likes
3
Public
Click a slice to open those files.
.gguf5.2 GB · 100%
From the Hugging Face model README
This is a normal llama.cpp GGUF quant of DavidAU's Deckard Gemma model:
The goal was a small, practical creative/voice/writing model that keeps more coherence than a plain Q4-style quant while still fitting comfortably on modest GPUs. The storyMaker name reflects the actual niche where it stood out: expressive writing, roleplay, voice-adjacent prose, and creative instruction following.
This was an unplanned release. The author was testing models for creative writing use, found a quantized GGUF of this model that showed severe looping and quality degradation, and built this version to check whether the issue came from the core finetune or from the quantization. The result was unexpectedly useful: creative, able to reason, good at following instructions, strong for roleplay, and expressive in a way that made it enjoyable to use. Because of that, it was uploaded for anyone else interested in this niche.
Because this was exploratory rather than a planned release, it has not gone through the author's usual comprehensive release process. A gender-affirming finetune based on the DavidAU model may come later for people interested in that direction.
This is not a plain one-shot Q4 quant.
The released GGUF is:
IQ4_NLq, k, v, and output projections in blocks 36..41Q8_0In other words, most of the model stays Q4-sized, while the final six attention blocks are kept at higher precision. The intent is to spend a small amount of extra size on the late attention path, where coherence, style continuity, and repetition stability are more likely to matter for writing-style use.
Main model:
Gemma-4-E4B-Deckard-AttnQ8-Last6-IQ4_NL-Imatrix-CreativeVoice.gguf
Optional template used in our deployment:
gemma4_deckard_no_think.jinja
The GGUF works with standard llama.cpp. No custom fork is required.
Example server command:
llama-server \
-m Gemma-4-E4B-Deckard-AttnQ8-Last6-IQ4_NL-Imatrix-CreativeVoice.gguf \
-ngl 999 \
-fa on \
-sm none \
-ctk f32 \
-ctv f32 \
--kv-offload \
-c 120000 \
-np 1 \
-b 512 \
-ub 128 \
--jinja \
--chat-template-file gemma4_deckard_no_think.jinja \
--reasoning off \
--reasoning-format deepseek \
--temp 0.9 \
--top-p 0.92 \
--min-p 0.03 \
--repeat-penalty 1.15 \
--repeat-last-n 256 \
--presence-penalty 0.10 \
--frequency-penalty 0.05 \
--dry-multiplier 0.60
The included no-thinking template is optional, but it is the one we used for the creative deployment. The upstream model is a thinking model, and in our creative tests the no-thinking template reduced scaffold-style answers.
The Q8 override rule was:
^blk\.(3[6-9]|4[0-1])\.attn_(q|k|v|output)\.weight=q8_0
That means 24 attention tensors were manually promoted from IQ4_NL to Q8_0:
blk.36..41.attn_q.weightblk.36..41.attn_k.weightblk.36..41.attn_v.weightblk.36..41.attn_output.weightQuantizer log summary:
model size = 14340.66 MiB, 16.00 BPW
quant size = 4980.91 MiB, 5.56 BPW
Released GGUF size:
5,238,686,720 bytes
These are direct runtime probes for this exact released GGUF using standard llama.cpp. No custom fork was used.
| GPU | Runtime | Context / KV | Prompt workload | Decode workload | Prompt tok/s | Decode tok/s | VRAM note |
|---|---|---|---|---|---|---|---|
| RTX 5060 Ti | llama.cpp b8763 server | -c 120000, f32 K/V, all layers offloaded | 1352 tokens | 256 tokens | 2816.56 | 73.54 | about 7796 MiB observed |
| RTX 5090 | llama.cpp b8763 server | -c 120000, f32 K/V, all layers offloaded | 1122 tokens | 40 tokens | 4251.93 | 138.82 | 7159 MiB llama.cpp self allocation |
5060 Ti log summary:
CUDA0 model buffer size = 2775.92 MiB
non-SWA KV buffer size = 3752.00 MiB
SWA KV buffer size = 60.00 MiB
compute buffer size = 572.00 MiB
5090 log summary:
CUDA0 self allocation = 7159 MiB
model buffer = 2775 MiB
context buffer = 3812 MiB
compute buffer = 572 MiB
This quant was tuned for English creative writing, expressive prose, conceptual writing, emotional/voice-adjacent discussion, and general reasoning. It was not built as a coding specialist.
Because the source model is an uncensored/aliterated model, apply your own safety, policy, and deployment filters where needed.
DavidAU/gemma-4-E4B-it-The-DECKARD-Claude-Opus-Expresso-Universe-HERETIC-UNCENSORED-ThinkingIQ4_NL.36..41 to Q8_0.llama.cpp.llama.cpp; it is not a safetensors checkpoint.