Downloads · 30 days
3.7K
55% of all-time downloads
sleepyeldrazi/Ornith-1.5-REAP-20B-A3B-GGUF
Ornith-1.5-REAP-20B-A3B-GGUF is a machine learning model from sleepyeldrazi. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as mit.
Experimental. An expert-pruned, recovery-fine-tuned, domain-calibrated-quantized slice of ornith-ai/Ornith-1.5-35B-A3B (MIT). 35.9B total → 20.3B total, ~3B active (256 → 132 routed experts per layer, 48.4% expert spa…
Downloads · 30 days
3.7K
55% of all-time downloads
All-time downloads
6.9K
Public
Repo size
153 GB
Likes
4
Public
Click a slice to open those files.
.gguf115 GB · 75%
From the Hugging Face model README
Experimental. An expert-pruned, recovery-fine-tuned, domain-calibrated-quantized slice of
ornith-ai/Ornith-1.5-35B-A3B (MIT). 35.9B total → 20.3B total, ~3B active (256 → 132 routed
experts per layer, 48.4% expert sparsity). Built with the REAP method, a light LoRA recovery pass,
and quantizations calibrated on agentic-coding / ML-engineering data.
⚠️ This is a research artifact. Expert pruning removes knowledge. Performance was validated around agentic coding and ML engineering (Python/C++). General-domain behavior (translation, creative writing, broad knowledge, multilingual nuance, long-document prose) has not been tested and is expected to be worse than the 35B base.
| file | what | size |
|---|---|---|
BF16/ | full-precision (BF16) reaped+recovered checkpoint | 36 GB |
GGUFs/Ornith-1.5-REAP-20B-A3B-UD-Q5_K_M.gguf | recommended — Q5_K_M + UD-style important-tensor raises, domain imatrix | 14.3 GB |
GGUFs/Ornith-1.5-REAP-20B-A3B-Q5_K_M.gguf | Q5_K_M, domain imatrix, no tensor map | 13.6 GB |
GGUFs/Ornith-1.5-REAP-20B-A3B-UD-IQ4_NL.gguf | IQ4_NL + IQ3_S/IQ4_NL expert map (smallest) | 10.5 GB |
GGUFs/recovered-f16.gguf, GGUFs/reap-only-f16.gguf | F16 reference GGUFs (pre/post recovery) | 38.1 GB each |
For IQ quantizations of the un-pruned 35B base, see AtomicChat/Ornith-1.5-35B-A3B-GGUF (the AD-IQ3_S/IQ3_XXS we measured against in Part 2 comes from there).
ud_iq4nl): the IQ3_S/IQ4_NL expert map costs ~0.26 PPL on general text —
fine for experimentation, not the quality pick.In multi-compaction, long-horizon, difficult ML-engineering tasks, two near-loop episodes have been observed (the model circling on "Let me reconsider" fallback tokens in its reasoning; Liquid-AI-style doom-loop mechanism #1). Both escaped on its own after ~10k tokens of looping — there is no evidence yet of the model being unable to escape by itself. If you see time-wasting loops in your own workload, these two flags are a very light nudge:
llama-server ... --repeat-penalty 1.01 --repeat-last-n 256
# or via the API sampling params: "repeat_penalty": 1.01, "repeat_last_n": 256
Keep repeat_last_n around 256 so legitimately repeated identifiers in code are not penalized;
the effect on normal output is negligible.
In limited personal testing, on ML-engineering tasks, q5k_ud (14.3 GB) outperformed
IQ3-level quantizations of Qwen3.6-35B and of Ornith-1.5-35B at roughly the same file size.
Be clear-eyed about what that means: in full precision, the reaped 20B probably does not
outperform the 35B bases — it is a domain-focused, high-precision slice of one of them, sized
for a workflow that wouldn't otherwise fit.
| quant | bits/weight (approx) | size |
|---|---|---|
| q5k_ud | ~5.5 (experts) + Q8_0 raises (attn/ssm/shared) | 14.3 GB |
| q5k_plain | ~5.5 | 13.6 GB |
| ud_iq4nl | ~4.1 (experts IQ3_S/IQ4_NL) | 10.5 GB |
S_j = mean(g_j · ||f_j||₂) over the calibration tokens; 256 → 132 experts/layer
(48.4%), top-8 routing preserved. Calibration used the domain mix above (code/ML-eng 35%,
tool 25%, agentic 20%, math+general 20%). Includes the router-weight renormalization fix
from the Cerebras repo.Related work referenced: LFM2 Technical Report (arXiv:2511.23404) — the tempered decoupled Top-K teacher loss (used conceptually for the recovery/distillation thinking), Liquid's Quantization-Aware Distillation (blog), NVIDIA ModelOpt NVFP4 (the serving path used for trace generation), and Unsloth Dynamic 3.0 (docs).
Same 382 KB held-out text (chat-format mixed domains, not in any calibration corpus), 25 × 4096-token chunks, greedy, llama.cpp, Q8_0 35B as reference:
| model | size | PPL | Δ vs 35B-Q8 |
|---|---|---|---|
| Ornith-1.5-35B (original, Q8_0) | 36.2 GB | 3.6850 | — |
| Ornith-1.5-35B AD-IQ3_S/IQ3_XXS<br>(AtomicChat GGUF, measured here) | 15.5 GB | 3.7603 | +0.075 |
| REAP-20B, pre-recovery (F16) | 38.1 GB | 4.3411 | +0.656 |
| REAP-20B, recovered (F16) | 38.1 GB | 4.3411 | +0.656 |
| q5k_ud (recovered, Q5_K_M+map) | 14.3 GB | 4.2730 | +0.588 |
Readings:
203 fixed prompts: tool 75 / code 45 / math 30 / abstention probes 30 / doom-prone 23 (temp 0.3, up to 8k tokens, 64-way):
| metric | recovered | pre-recovery |
|---|---|---|
| completed rows | 87% (177/203) | 87% (176/203) |
| honest abstention on probes | 10% (3/30) | 7% (2/30) |
| doom-loop n-gram overlap | 0.026 | 0.029 |
| avg tool length | 13.8k chars | 14.1k chars |
Both are effectively indistinguishable on these coarse axes — the recovery LoRA neither regressed nor visibly improved generic behavior; its value is in-domain fluency.
| variant | scheme | size | PPL |
|---|---|---|---|
| q5k_ud | Q5_K_M + UD-style map | 14.3 GB | 4.2730 |
| q5k_plain | Q5_K_M | 13.6 GB | 4.3146 |
| q6k_plain | Q6_K | 15.7 GB | 4.3021 |
| ud_iq4nl | IQ4_NL + IQ3_S map | 10.5 GB | 4.6038 |
--no-mtp); no speculative decoding.ssm_out (linear-attention state output) is kept at Q8_0 (per Unsloth's Qwen3.5-family
guidance, quantizing it hurts).Qwen3_5MoeForCausalLM, 132
experts, text-only config) — use it to re-quantize with different maps/imatrixes.