Downloads · 30 days
38
5% of all-time downloads
undeadindustries/gemma-4-26b-a4b-it-nvfp4-hybrid-smlp
gemma-4-26b-a4b-it-nvfp4-hybrid-smlp is a text generation model from undeadindustries. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Instruction-following–first hybrid NVFP4 of google/gemma-4-26B-A4B-it.
Downloads · 30 days
38
5% of all-time downloads
All-time downloads
758
Public
Parameters
2.4B
18.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors17.6 GB · 100%
How the weights are stored.
U811.4B · 72%
From the Hugging Face model README
Instruction-following–first hybrid NVFP4 of google/gemma-4-26B-A4B-it.
Canonical undeadindustries Gemma 4 26B A4B hybrid (~16.4 GiB, 32 GB GPU target). The earlier “shared MLP NVFP4” hybrid and its drafter were removed after this recipe proved better on structured IF workloads.
Built and validated on DGX Spark (GB10, 128 GB UMA). Intended deployment: single 32 GB GPU, TP=1.
| Component | Precision |
|---|---|
Attention (q/k/v/o) | BF16 |
| MoE router | BF16 |
Shared dense MLP (gate/up/down) | BF16 |
| MoE expert linears (128×8) | NVFP4 W4A4 |
Keeping the shared MLP in BF16 (every token hits it) improves outline hierarchy, ordered steps, heading discipline, and multi-source disambiguation vs quantizing that path to NVFP4, while experts stay NVFP4 so disk/VRAM stay ~16 GB.
AutoRound flag: AR_PROTECT_SHARED_MLP=1 with bare layer_config keys mlp.gate_proj, mlp.up_proj, mlp.down_proj (not pre-escaped regex — see project Bug X).
| Item | Value |
|---|---|
| Total params | ~25.2B |
| Active params / token | ~3.8B (A4B) |
| Layers | 30 (hybrid 5-sliding : 1-global) |
| MoE | 128 experts, top-8 routed + shared dense MLP |
| Text-only | Vision stripped; Gemma4ForCausalLM |
Greedy /v1/chat/completions on vLLM 0.20.2:
###/##### hierarchy, SM64 no-JSON/no-DS, elaborative density floor): 0 failures on the in-repo batteryUse the chat API for -it models — not raw /v1/completions.
vllm serve undeadindustries/gemma-4-26b-a4b-it-nvfp4-hybrid-smlp \
--port 8000 \
--max-model-len 8192 \
--gpu-memory-utilization 0.90 \
--max-num-seqs 8 \
--max-num-batched-tokens 8192 \
--enforce-eager \
--trust-remote-code \
--generation-config vllm
Requires vLLM ≥ 0.20.2 with NVFP4 + compressed-tensors support.
vllm serve undeadindustries/gemma-4-26b-a4b-it-nvfp4-hybrid-smlp \
--port 8000 --max-model-len 8192 \
--gpu-memory-utilization 0.90 --max-num-seqs 8 \
--max-num-batched-tokens 8192 \
--enforce-eager --trust-remote-code --generation-config vllm \
--speculative-config '{"model":"undeadindustries/gemma-4-26b-a4b-it-dflash-drafter-nvfp4-smlp","method":"dflash","num_speculative_tokens":15}'
Use the paired smlp drafter above — a BF16-base or mismatched-verifier drafter gets ~0% acceptance.
scheme=NVFP4, iters=200, nsamples=256, llm-compressor exportDerived from google/gemma-4-26B-A4B-it (Apache 2.0).
Subject to Gemma Terms of Use.