Ornith-1.5-9B-MTP-NVFP4-newhead

Ornith-1.5-9B MTP-NVFP4 — new official head grafted on protoLabs NVFP4 body

Head-graft build: the FP4 body from protoLabsAI/Ornith-1.5-9B-MTP-GGUF (-MTP-NVFP4.gguf, 23 Aug snapshot) with the 15 blk.32 MTP-head tensors replaced by the updated official ones from ornith-ai/Ornith-1.5-9B-GGUF (-Q8_0.gguf, 24 Aug MTP-weights update). Binary splice — tensor names, types (Q8_0/F32) and sizes match byte-identically, so no requantization was needed.

Why

The 24 Aug ornith-ai update improved the MTP head significantly, but no GGUF NVFP4 build carries it yet. This graft gets both: Blackwell-native FP4 kernels (the body) + the new head.

Benchmarks (RTX 5080 16GB, llama.cpp PR #27342 build, 65K ctx)

Quality spot-checks: SimCity one-shot 8/9 (23.6K chars), tool-calling 3/3 with parallel calls.

Usage

llama-server --model Ornith-1.5-9B-MTP-NVFP4-newhead.gguf \
  --n-gpu-layers 99 -fit off --ctx-size 65536 --flash-attn on --jinja \
  --temp 1.0 --top-k 20 --top-p 0.95 --min-p 0.0 --presence-penalty 1.5 \
  --reasoning-budget 4096 \
  --spec-type draft-mtp --spec-draft-n-max 3

Notes:

• -fit off + explicit -ngl are mandatory — MTP inverts if any layer lands on CPU
• --spec-draft-n-max 3 maximizes throughput (2 = acceptance, 4 regresses)
• --reasoning-budget recommended, the xhigh think block otherwise eats the whole max_tokens
• Vision: pair with the official mmproj-Ornith-1.5-9B-BF16.gguf from ornith-ai
• Requires a recent llama.cpp build with NVFP4 GGUF support (Blackwell GPUs)

Graft recipe

# for each of the 15 blk.32.* tensors (same names/types/sizes in both files):
#   copy bytes at data_offset/n_bytes from official Q8 into a copy of the NVFP4

Credits & license

• Body: protoLabsAI (MIT) — NVFP4 GGUF packaging
• Head: ornith-ai (MIT) — official MTP weights, 24 Aug update
• Model: ornith-ai/Ornith-1.5-9B (MIT)