Downloads · 30 days
14
5% of all-time downloads
KaviarLabs/Ornith-1.0-35B-UD-Q4_K_XL-MTP-GGUF
Ornith-1.0-35B-UD-Q4_K_XL-MTP-GGUF is a text generation model from KaviarLabs. Use it when you need the model to write or continue text. It is set up for gguf.llama.cpp. The card lists the license as mit.
This repository contains an experimental MTP-enabled build of Ornith-1.0-35B based on Unsloth's UD-Q4KXL GGUF.
Downloads · 30 days
14
5% of all-time downloads
All-time downloads
282
Public
Repo size
22.9 GB
Likes
0
Public
Click a slice to open those files.
.gguf22.9 GB · 100%
From the Hugging Face model README
This repository contains an experimental MTP-enabled build of Ornith-1.0-35B based on Unsloth's UD-Q4_K_XL GGUF.
The normal Ornith model weights and Unsloth quantization map are preserved. The only model-weight addition is a set of 20 MTP tensors (blk.40.*) grafted from the Q4_K_M donor-head package published by skinnyctax. The resulting model can use llama.cpp's draft-mtp self-speculative decoding without loading a separate draft model.
| File | Size | SHA-256 |
|---|---|---|
Ornith-1.0-35B-UD-Q4_K_XL-MTP.gguf | 21.29 GiB (~22.87 GB) | 366d7ee1e2322fba81308f7116cd0263e4acde855224d215149cf24970009e65 |
Ornith-1.0-35B-UD-Q4_K_XL.ggufThe base GGUF contains the complete Ornith model with Unsloth's UD-Q4_K_XL mixed quantization layout and tokenizer metadata.
ornith-1.0-35b-Q4_K_M-MTP-donor-heads.ggufgguf_mtp_graft.pyThe donor file is not a standalone draft model. It contains only the MTP head tensors intended to be grafted into a compatible 40-block qwen35moe GGUF.
The build started from:
Ornith-1.0-35B-UD-Q4_K_XL.gguf
and grafted the Q4_K_M MTP donor heads:
ornith-1.0-35b-Q4_K_M-MTP-donor-heads.gguf
The transformation was performed with the donor repository's gguf_mtp_graft.py logic.
Before grafting, the Unsloth target contained:
53 metadata KV pairs
733 tensors
qwen35moe.block_count = 40
The donor contributed:
20 MTP tensors
blk.40.*
The final GGUF contains:
54 metadata KV pairs
753 tensors
qwen35moe.block_count = 41
qwen35moe.nextn_predict_layers = 1
llama.cpp interprets this as:
n_layer = 40
n_layer_all = 41
In other words, the original 40 model layers remain the target model and the appended layer is used as the MTP prediction head.
The tokenizer from the Unsloth base GGUF was preserved intact. Validation showed:
vocab type = BPE
n_vocab = 248320
n_merges = 247587
This matters because the standalone donor-head GGUF intentionally does not contain the full tokenizer arrays and therefore cannot be loaded directly as --model-draft by current llama.cpp builds.
The version of gguf_mtp_graft.py retrieved during this build required a small local correction when re-serializing general.architecture.
The published script stored that value with:
kv_pairs.append((key, 8, arch.encode("utf-8")))
For the GGUF string value to retain its length-prefixed encoding, this build used:
kv_pairs.append((key, 8, enc_str(arch)))
Without that correction, the generated file failed GGUF header parsing in the llama.cpp build used for validation. No model tensor data was changed by this fix; it only corrected serialization of that metadata string during header reconstruction.
This file should be understood as Unsloth UD-Q4_K_XL plus Q4_K_M MTP heads, not as a globally uniform Q4_K_M model.
UD-Q4_K_XL GGUF.The validated final tensor-type summary was:
f32 : 368 tensors
f16 : 2 tensors
q8_0 : 252 tensors
q4_K : 86 tensors
q5_K : 39 tensors
q6_K : 6 tensors
The final GGUF was validated with:
llama.cpp build: b10331
commit: 7ba604f1c
platform: Windows AMD64
GPU: NVIDIA GeForce RTX 4070
The model successfully loaded with:
--spec-type draft-mtp--spec-draft-n-max 3mmproj-F16.ggufllama.cpp successfully created the MTP draft context against the target model, initialized speculative decoding, loaded the multimodal projector, and reached model loaded / listening state.
Do not use the donor-head GGUF as --model-draft. The MTP heads are already embedded in this model.
A minimal invocation is:
llama-server \
--model Ornith-1.0-35B-UD-Q4_K_XL-MTP.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 3
A configuration close to the one used during validation is:
llama-server \
--model Ornith-1.0-35B-UD-Q4_K_XL-MTP.gguf \
--mmproj mmproj-F16.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 3 \
--host 127.0.0.1 \
--port 8080 \
--threads-http 4 \
--device CUDA0 \
--n-cpu-moe 36 \
--gpu-layers 999 \
--ctx-size 131072 \
--parallel 1 \
--flash-attn on \
--cache-type-k q8_0 \
--cache-type-v q8_0 \
--spec-draft-type-k q8_0 \
--spec-draft-type-v q8_0 \
--batch-size 4096 \
--ubatch-size 512 \
--threads 8 \
--threads-batch 16 \
--jinja \
--load-mode mlock \
--fit off
For multimodal/vision use, obtain a compatible projector such as Unsloth's mmproj-F16.gguf from the original GGUF repository. The projector is not embedded in this model file.
The donor repository reports substantial MTP acceptance and inference-speed improvements for its own Q4_K_M and Q6_K grafted builds. Those measurements should not be assumed to apply directly to this UD-Q4_K_XL build.
This specific file has been functionally validated for correct loading and MTP initialization, but no controlled acceptance-rate or tokens-per-second benchmark is claimed here yet.
UD-Q4_K_XL quantizationdraft-mtp speculative decoding supportThis repository does not claim authorship of the original model, quantization method, or donor MTP work. It packages the specific UD-Q4_K_XL + MTP graft described above for convenient llama.cpp use.