Downloads · 30 days
1.1K
13% of all-time downloads
gbuzhf/Ornith-1.5-35B-A3B-TIEL-Calibrated-MTPv2-ICE-GGUF
Ornith-1.5-35B-A3B-TIEL-Calibrated-MTPv2-ICE-GGUF is a text generation model from gbuzhf. Use it when you need the model to write or continue text. The card lists the license as mit.
Four GGUF tiers of the original ornith-ai/Ornith-1.5-35B-A3B, with ornith-ai's trained MTPv2 head embedded, calibrated on Tiel's importance matrix, carrying the Qwen-Sharp v22.4.1 chat template.
Downloads · 30 days
1.1K
13% of all-time downloads
All-time downloads
8.6K
Public
Repo size
87.4 GB
Likes
10
Trending 1
Click a slice to open those files.
.gguf87.4 GB · 100%
From the Hugging Face model README
Four GGUF tiers of the original ornith-ai/Ornith-1.5-35B-A3B, with ornith-ai's
trained MTPv2 head embedded, calibrated on Tiel's importance matrix, carrying the Qwen-Sharp v22.4.1 chat template.
| tier | size | mean KLD | vs the Unsloth Dynamic ladder |
|---|---|---|---|
23G-ICE | 22.84 GB | 0.0325 | Beats UD-Q4_K_XL on both axes — 14.5 % closer to bf16 and 0.37 GB smaller. Best value here. |
25G-ICE | 24.85 GB | 0.0284 | The best file below 25 GB — nothing smaller is closer to bf16. Also beats APEX-I-Balanced (26.28 GB / 0.0345) by 12 % while being 1.4 GB smaller. Above it, UD-Q5_K_S at 25.83 GB is stronger. |
21G-ICE | 20.85 GB | 0.0389 | Fills a 4.5 GB hole in the UD ladder. Essentially UD-Q4_K_XL quality (0.0380) at 2.36 GB less. |
19G-ICE | 18.82 GB | 0.0601 | 17 % closer to bf16 than UD-IQ4_XS (0.0723) for +0.14 GB. |
Short version: ICE wins in the 19–23 GB band and loses above 25 GB — see Method.
KL divergence against the BF16 master this repo was built from. WikiText-2 raw test,
64 chunks, n_ctx 2048, one binary and one reference for all four files.
Mean PPL(base) = 7.494953 ± 0.079236.
Sorted best → worst by overall (BF16 = 100), the same composite used on the other
Ornith-1.5 cards: 0.70/(1+meanKLD) + 0.30*sameTop1.
| tier | size | mean KLD | 99% KLD | 99.9% KLD | PPL ratio | same top-1 | active bpw | file bpw | overall |
|---|---|---|---|---|---|---|---|---|---|
25G-ICE | 24.84 GB | 0.0284 | 0.286 | 1.120 | 0.9851 | 93.44% | 7.686 | 5.597 | 96.1 |
23G-ICE | 22.83 GB | 0.0325 | 0.335 | 1.177 | 0.9900 | 92.85% | 7.523 | 5.143 | 95.7 |
21G-ICE | 20.84 GB | 0.0389 | 0.419 | 1.584 | 1.0001 | 92.24% | 7.357 | 4.695 | 95.1 |
19G-ICE | 18.82 GB | 0.0601 | 0.616 | 2.387 | 1.0013 | 90.33% | 7.192 | 4.240 | 93.1 |
Active bpw weights each tensor by how often it actually runs — routed experts at
k/E — so it says where the bits went in the forward pass rather than on disk. It
explains a design; it does not rank one. Ranking is on measured KLD.
Every GGUF quantizer — llama.cpp's own mixes, Unsloth Dynamic, APEX — minimises the same thing for every tensor: importance-weighted error of that tensor's output, for the current token. That is correct for a tensor whose error dies with the token, and wrong for the ones whose error does not.
ICE sorts tensors by how far an error travels, then pays accordingly:
| class | what it is here | why | type |
|---|---|---|---|
| discrete | the MoE router | an error flips an argmax, so a different expert runs. Not a graded loss — a categorical one. | F32 |
| recurrent | SSM decay / timestep terms | the error enters a carried state and compounds along the sequence | F32 |
| cached | attn_k, attn_v | written to the KV cache once, re-read by every later token, never re-decided | F16 |
| instant | everything else, incl. all 256 experts | the error affects this token only | the dial |
The first three classes are 0.14 % of the model — 47 M parameters, 0.15 GB.
Freezing them outright is a line item, not a trade-off. The recovered budget plus the
whole remaining budget goes to the expert bank, uniformly across gate/up/down,
with the higher type placed shallow-first.
Where the budget goes furthest. All four tiers here are Pareto-optimal against a twelve-tier comparison of the Unsloth Dynamic and APEX ladders on this model, measured on one harness against one reference: nothing published is both smaller and closer to bf16 than any of them.
The advantage is largest in the 19–23 GB band and narrows as the budget rises: above
~25 GB the expert term saturates and what remains is the dense path, which is the
regime UD's "pin the dense path high" policy is built for. That is why UD-Q5_K_S and
UD-Q6_K remain the strongest files on the board, and why this ladder stops at 25 GB
rather than chasing them.
Full report, including seven negative results and the retraction of a rule this work itself derived and shipped: gbuzhf/ICE-quantization
Ornith-1.5-35B-A3B-TIEL_Calibrated-MTPv2-{19,21,23,25}G-ICE.gguf
Each is a single file carrying the MTPv2 head (block_count=41, 753 tensors) and the
Sharp template. For self-speculative decoding pass --spec-type draft-mtp; the head
is embedded, no sidecar model is needed
The GGUFs embed Qwen-Sharp v22.4.1, which stays the default. For agent harnesses (tool use, multi-step work), the repo also ships arrow4.0 — opt in by passing the file:
llama-server -m <model>.gguf --jinja --chat-template-file chat_template_sharp-v22.5.0-arrow4.0.jinja
arrow4.0 is our agent layer on Qwen-Sharp v22.5.0. Plain chat stays byte-identical to Sharp; with tools it adds short agent rules, one note per tool result, a STATUS line, thinking only at decision points, .arrow project memory and a hand-back report when work stalls.
Full description and switches: gbuzhf/Qwen-Sharp-Arrow-Chat-Templates.