Downloads · 30 days
51
7% of all-time downloads
Danny-Dasilva/gemma-4-12B-it-antidoom-W4A16-GPTQ-g32-DSpark
gemma-4-12B-it-antidoom-W4A16-GPTQ-g32-DSpark is a text generation model from Danny-Dasilva. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as gemma.
A Antidoom (FTPO anti-repetition) version of google/gemma-4-12B-it, quantized post-training to GPTQ int4 (W4A16, group=32, symmetric) and verified as a DSpark speculative-decoding target
Downloads · 30 days
51
7% of all-time downloads
All-time downloads
685
Public
Parameters
12.3B
8.3 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors8.3 GB · 100%
How the weights are stored.
I3210.9B · 89%
From the Hugging Face model README
A Antidoom (FTPO anti-repetition) version of
google/gemma-4-12B-it, quantized post-training to
GPTQ int4 (W4A16, group=32, symmetric) and verified as a
DSpark speculative-decoding target
(draft head: deepseek-ai/dspark_gemma4_12b_block7).
TL;DR: 207 tok/s single-stream on one RTX 5090 (32 GB) — the fastest gemma-4-12B configuration we have measured on that card (beats our non-antidoom GPTQ at 198), at 7.8 GB of weights.
DSpark acceptance survived and improved: 35.6% (non-antidoom GPTQ) → 38.3% (this model). The FTPO patch suppresses exactly the degenerate repetition the draft also fails on, so speculation gets slightly better.
| config | tok/s | DSpark accept |
|---|---|---|
| this model + DSpark k=7 + CUDA graphs + autotune | 207.5 (peaks 260) | 38.3% |
| this model, native (no speculation) | 119 | — |
| reference: non-antidoom GPTQ + DSpark k=7 + autotune | 198.2 | 35.6% |
| reference: official QAT w4a16-ct + DSpark | slower than native | 2.1% |
from vllm import LLM, SamplingParams
from transformers import AutoTokenizer
MODEL = "Danny-Dasilva/gemma-4-12B-it-antidoom-W4A16-GPTQ-g32-DSpark"
DRAFT = "deepseek-ai/dspark_gemma4_12b_block7"
llm = LLM(
model=MODEL,
max_model_len=8192,
attention_backend="FLASHINFER", # gemma-4 full-attn layers have head_dim 512
enable_flashinfer_autotune=True, # +2% in our runs
speculative_config={
"method": "dspark",
"model": DRAFT,
"num_speculative_tokens": 7, # = draft block size; k<7 or k=14 are both slower
"attention_backend": "TRITON_ATTN",
},
)
tok = AutoTokenizer.from_pretrained(MODEL)
prompt = tok.apply_chat_template(
[{"role": "user", "content": "Explain transformers step by step."}],
add_generation_prompt=True, tokenize=False,
)
print(llm.generate([prompt], SamplingParams(temperature=0.0, max_tokens=256))[0].outputs[0].text)
Notes:
<|turn>-style tokens and
add_bos_token=False; raw prompts produce garbage on any backend.num_speculative_tokens must be ≤ 7 or a multiple of 7 (draft block size). 7 is optimal.kv_cache_dtype="fp8" measured slower (−3%) at short context on this card.llm-compressor 0.12 GPTQ, one-shot:
HuggingFaceH4/ultrachat_200k (train_sft), chat template applied,
max_seq_length 2048pack-quantized format —
mirrors the official QAT checkpoint's config so every vLLM kernel path is identicalsequential_targets=["Gemma4UnifiedTextDecoderLayer"]; lm_head and vision/audio embedder
projections kept in bf16 (same 17-entry ignore list as the official QAT release, spelled so both
HF and vLLM module names match)processor_config.json included (vLLM's multimodal processor init requires it)google/gemma-4-12B-it @ 5926caa4 → antidoom FTPO LoRA merge → GPTQ W4A16 g32