Ornith-1.5-35B-A3B-APEX-GGUF

Ornith-1.5-35B-A3B — APEX GGUF

Three banded-allocation GGUF tiers of ornith-ai/Ornith-1.5-35B-A3B — 35.5 B total / ~3 B active MoE, 256 routed experts at top-8, hybrid gated-delta-net with full attention every 4th layer, 262 K context, plus a 1-layer MTP head.

These differ from a flat Q4_K_M in three specific ways: routed experts are banded by depth rather than given one type; the 60 recurrent state/gate coefficients are pinned F32; and the MTP head is retained rather than dropped.

Files

Shared experts are Q8_0 and attention is Q6_K in every tier. bpw is over all 35.505 B parameters, computed from the file size.

Start with i-quality. i-compact and i-mini exist for smaller memory budgets; only i-quality has been benchmarked (below).

For the vision projector, use the one from the source publisher — we did not rebuild or re-host it: mmproj-Ornith-1.5-35B-BF16.gguf. It works with i-quality on stock llama.cpp — measured, see below.

Running it

llama-server \
  --model Ornith-1.5-35B-A3B-APEX-i-quality.gguf \
  -ngl 999 -fa on -c 131072 \
  --jinja --temp 1.0 --top-k 20 --top-p 0.95

temp 1.0 / top_k 20 / top_p 0.95 is the source model's own default, carried in the GGUF metadata. Substituting another model's sampling changes results materially.

KV cache is cheap: only 10 of the 40 backbone layers are full attention (2 KV heads, head_dim 256), so 20 KiB/token — 2.5 GiB at 131072, 5.0 GiB at the full 262144. The other 30 layers hold a constant-size recurrent state.

The MTP head

The source checkpoint's 1-layer multi-token-prediction block (blk.40) is kept, quantized flat at Q4_K (11 tensors) with its norms at F32. Most GGUF conversions of this model family drop it.

Q4_K is not arbitrary: on GLM-4.7-Flash, Q4_K measured the best size/speed trade, and IQ2 was measurably worse. Mainline llama.cpp consumes the head via --spec-type draft-mtp, passing this same file as its own draft model:

llama-server -m Ornith-1.5-35B-A3B-APEX-i-quality.gguf \
  --spec-type draft-mtp -md Ornith-1.5-35B-A3B-APEX-i-quality.gguf \
  --spec-draft-n-max 1 \
  -ngl 999 -fa on -c 131072 --parallel 1 --temp 1.0 --top-k 20 --top-p 0.95

Use --spec-draft-n-max 1. Measured on i-quality, DGX Spark (GB10), stock llama.cpp, 300 tokens at temp 1.0, fixed seed, cache_prompt: false:

The two no-drafter runs bracket the run-to-run spread at about 1%. Acceptance falls from 0.674 to 0.133 as depth grows while mean accepted length stays near 1.7, so depths above 1 add draft cost without adding accepted tokens. Depth does not transfer between models — measure it.

What was measured

i-quality, agentic coding suite, frontier tier (9 tasks), one run, DGX Spark (GB10), stock llama.cpp, sampling as above:

Vision

i-quality plus the publisher's projector, stock llama.cpp, five perception tasks put to the server directly (read a placard, transcribe a handwritten spec, name a function and its buggy line from a screenshot, read a value off a bar chart, count shapes):

The second row is the control: every answer is unguessable by construction (a coined phrase, a coined identifier, an arbitrary count, a deliberately non-standard base case), so a blind pass would mean the task was measuring text priors rather than the projector. None passed blind.

Speed

Generation speed, DGX Spark (GB10), stock llama.cpp, 8192 ctx, single slot:

i-compact and i-mini were load-and-generate checked only; the two harnesses differ, so read across tiers as approximate.

Read that as a floor, not a ranking: this suite saturates. A clean sweep means the quant cleared the competence bar, not that it beats something else. The discriminating measurement once score saturates is output tokens at equal score, and that needs several runs per arm — this is one run, on a suite where individual tasks are known to flip between identical runs.

Provenance and sources

• Source weights: ornith-ai/Ornith-1.5-35B-A3B safetensors, converted to BF16 GGUF here.
• Importance matrix: not ours. These quants were built against bartowski's published imatrix for this model (sha256 8d5b1693…c0228), used unmodified. Credit for the calibration prior belongs there; what is ours is the allocation across tensors.
• Allocation method: structural banding by depth against a size target, not measured per-tensor sensitivity. Ornith's architecture has no sensitivity name-map in our pipeline, so the allocator falls back to the structural prior.
• Vision projector: the publisher's own file, linked above rather than re-hosted.
• Unofficial community conversion — not affiliated with or endorsed by the source publisher. Provided as-is, without warranty.

Caveats

• Vision is verified on i-quality only, with the publisher's projector, on the five tasks above. i-compact and i-mini have not been checked for vision.
• Structural allocation is a prior, not a measurement. On at least one model where both paths could be compared, the structural prior did worse than stock allocation. It is used here because measured sensitivity is not yet available for this architecture.
• i-compact and i-mini are unbenchmarked. Both load and generate correct-looking code, and pass structural checks, but neither has been through the agentic suite.
• IQ2 tiers at 2.8 bpw are aggressive on a 3 B-active MoE; expect i-mini to degrade on long multi-step reasoning well before i-quality does.