Downloads · 30 days
152
2% of all-time downloads
skinnyctax/Ornith-1.0-35B-Q6_K-Frankenstein-MTP-GGUF
Ornith-1.0-35B-Q6_K-Frankenstein-MTP-GGUF is a text generation model from skinnyctax. Use it when you need the model to write or continue text. It is set up for gguf.llama.cpp. The card lists the license as mit.
Cross-model MTP head graft — speculative decoding on a model that was never trained with MTP heads.
Downloads · 30 days
152
2% of all-time downloads
All-time downloads
7.8K
Public
Repo size
52.2 GB
Likes
10
Public
Click a slice to open those files.
.gguf52.2 GB · 100%
From the Hugging Face model README
Cross-model MTP head graft — speculative decoding on a model that was never trained with MTP heads.
Available in Q6_K and Q4_K_M quantizations.
Ornith-1.0-35B is an agentic-coding MoE (35B-A3B, Qwen3.5 base) with excellent performance but no native MTP (multi-token prediction) support.
This GGUF has 20 MTP head tensors surgically grafted from a sibling finetune of the same architecture — Qwopus3.6-35B-A3B, which does ship with MTP heads. The donor and target share the same qwen35moe architecture, embedding dimensions, and hybrid SSM/Mamba layer structure, making the graft surprisingly effective.
The result: ~20-25% inference speedup via --spec-type draft-mtp self-speculation, with no quality degradation.
| Metric | Stock (no MTP) | Q6_K MTP | Q4_K_M MTP |
|---|---|---|---|
| Speed | 126 tok/s | 152 tok/s | 158 tok/s |
| MTP acceptance | — | 74.3% | 79.5% |
| VRAM (dual 3090) | 28 GB | 32 GB | 23 GB |
| Quality | Clean | Clean | Clean |
--spec-type draft-mtp --spec-draft-n-max 4Q4 achieves higher acceptance than Q6 — the quantization noise in the target model appears to align better with the MTP draft predictions. Q4 also runs faster (less memory bandwidth) and fits more comfortably in VRAM.
| File | Size | Description |
|---|---|---|
ornith-1.0-35b-Q6_K-MTP-final.gguf | 28 GB | Full Q6_K grafted model |
ornith-1.0-35b-Q4_K_M-MTP.gguf | 21 GB | Full Q4_K_M grafted model (smaller, faster, higher acceptance) |
ornith-1.0-35b-Q6_K-MTP-donor-heads.gguf | 0.69 GB | Q6_K MTP heads only — for DIY grafting |
ornith-1.0-35b-Q4_K_M-MTP-donor-heads.gguf | 0.55 GB | Q4_K_M MTP heads only — for DIY grafting |
gguf_mtp_graft.py | 7 KB | The graft surgery script — works with any same-architecture model pair |
Already have the base Ornith GGUF (or another qwen35moe model)? Skip the big download and graft locally in ~2 minutes:
# 1. Download the donor heads + graft script
hf download skinnyctax/Ornith-1.0-35B-Q6_K-Frankenstein-MTP-GGUF \
ornith-1.0-35b-Q6_K-MTP-donor-heads.gguf gguf_mtp_graft.py \
--local-dir .
# 2. Graft onto your base GGUF (Q6 example)
python3 gguf_mtp_graft.py \
your-base-model-Q6_K.gguf \
ornith-1.0-35b-Q6_K-MTP-donor-heads.gguf \
output-MTP.gguf
# 3. Run with draft-mtp speculation
llama-server --model output-MTP.gguf --spec-type draft-mtp --spec-draft-n-max 4 ...
Match donor quant to target quant for best results. Q6 heads on Q6 base, Q4 heads on Q4 base. Cross-quant grafting (e.g. Q6 heads on Q4 base) will load but may have lower acceptance.
The donor heads work on any qwen35moe model with 40 blocks (e.g. Ornith, Qwopus3.6, other Qwen3.5-A3B finetunes).
blk.40.*)qwen35moe.block_count: 40 → 41qwen35moe.nextn_predict_layers: added = 1The 20 MTP tensors are: attention (q/k/v/output norms), shared + 256-routed experts (gate/up/down), and the nextn projection heads — essentially one full transformer layer's worth of weights.
The Q4 variant was produced by requantizing the Q6 grafted model to Q4_K_M using llama.cpp's --allow-requantize flag. All 20 MTP tensors were preserved and converted to Q4.
llama-server \
--model ornith-1.0-35b-Q4_K_M-MTP.gguf \
--port 8080 \
--ctx-size 131072 \
--threads 16 \
--batch-size 1024 --ubatch-size 512 \
--n-predict 8192 \
--spec-type draft-mtp --spec-draft-n-max 4 \
--temp 0.6 --top-p 1.0 --top-k 20 \
--flash-attn on \
--cont-batching \
--kv-unified \
-ctk q8_0 -ctv q8_0 \
-ngl 999 --no-mmap --mlock \
--jinja --reasoning off
Single GPU (RTX 3090): Add --n-cpu-moe 12 to offload some expert layers to CPU. Q4 fits more easily than Q6.
Dual GPU: Use --tensor-split 50,50 --main-gpu 0 and everything fits on-GPU.
draft-mtp support (the --spec-type draft-mtp flag)