Ornith-1.5-9B-MTP-ASHQ1-GGUF

Ornith-1.5-9B / 1.0-9B — ASHQ1 Quantization (MTP-Aware)

ASHQ1 quantization of Ornith with explicit MTP (Multi-Token Prediction) handling — the whole MTP head (blk.32.* + nextn.*) is kept at Q8_0 and excluded from the classifier budget, while the priority queue optimises the main transformer layers by Σ(importance) × ΔMSE / MiB.

Note: File names contain "BF16" for HuggingFace parser compatibility — these are ASHQ1 quants, not BF16.

🧪 KLD-validated (battery by Soulfate24, ctx 512×64, Flash-Attention). PPL is a canary, KLD-vs-BF16 is the rank column. Honest placement: our 6500 is Nano-class fidelity at Quality-class size — Remix Quality-36 holds the rank here (KLD 0.0241 vs our 0.0763). PPL-only verdict lines retired per our own standing rule.

Quants — Ornith-1.5-9B (current)

Benchmark vs ASHQ1-Remix (same model; PPL = canary, KLD = rank)

Our 6500 takes the PPL column; Remix Quality takes the rank column. Both truths stay on the card.

Top-down experiment

--top-down reverses the greedy: every quantizable tensor starts at F16 — norms included — and is downgraded cheapest-loss-first until the budget fits. Result on 1.5 @6500: all 177 norm tensors sank to the Q4_K floor, zero tensors left at F16 — and PPL didn't move (8.6337 vs 8.6341, σ=0.06). Takeaway: norms on this architecture are free real estate — quantizing all of them to Q4_K costs nothing, but PPL is too coarse a metric to separate the two allocation paths. Sensitive tasks (HumanEval, MMLU) may tell them apart.

How ASHQ1 Works (v7)

1. Base: norms/ssm_params → F16, output/token_embd pinned → Q5_K (outside budget — llama.cpp writes them at the output/token types unconditionally), rest → Q4_K (--allow-q3-or-lower: CAN_Q3 types → IQ2_XXS).
2. MTP head (whole blk.32 layer, detected by excluding it from n_layers) → Q8_0, outside budget.
3. Priority queue over tied groups + singletons: Σ(importance) × ΔMSE / MiB, theoretical MSE metric (ΔMSE = 2^{−2·bpw₁} − 2^{−2·bpw₂}, K-quants +0.1 effective bpw). Zero-cost steps (IQ4_NL→Q4_K) apply as free upgrades inline.
4. Drain best-first until the target fits. Estimator accuracy: ±3 MiB vs llama-quantize --dry-run.

Method, source code and older benchmarks: wepiqx/ASHQ1.

Usage

llama.cpp

# Chat mode with built-in Jinja template (recommended)
llama-cli \
  -m Ornith-1.5-9B-MTP-BF16-ASHQ1-6500.gguf \
  --jinja \
  -ngl 99 \
  -c 8192

Recommended sampling: temperature 0.6–1.0, top_p 0.95, top_k 20.

Ollama

The model's built-in Jinja template is auto-detected by Ollama — no need to specify TEMPLATE manually.

1. Create a Modelfile:

FROM ./Ornith-1.5-9B-MTP-BF16-ASHQ1-6500.gguf

PARAMETER num_ctx 8192
PARAMETER temperature 0.6
PARAMETER top_k 20
PARAMETER top_p 0.95

1. Build and run:

ollama create ornith-15-ashq1-6500 -f Modelfile
ollama run ornith-15-ashq1-6500

LM Studio

1. Open LM Studio
2. Drag the desired GGUF file into the app
3. Set GPU Offload to 99 layers
4. Set context length to 8192+
5. Set preset sampling: temperature 0.6, top_p 0.95, top_k 20
6. Start chatting

Quantization Config — 1.5 @6500

Generated by ASHQ1 (v7, current code). Reproduce with: python main.py --model Ornith-1.5-9B-MTP-BF16.gguf --imatrix Ornith-1.5-9B-imatrix.gguf --size 6500 --run

<details> <summary>llama-quantize arguments (click to expand)</summary>

--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(32)\.nextn=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.ffn_down=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.ffn_gate=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.ffn_up=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.post_attention_norm=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.attn_q_norm=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.attn_output=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.attn_norm=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.attn_k_norm=Q8_0"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q6_K"
--tensor-type "(blk|BLK)\.(3|7|23|27|(?:31|32))\.attn_q=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|23|27|(?:31|32))\.attn_k=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|23|27|(?:31|32))\.attn_v=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q8_0"
--tensor-type "(blk|BLK)\.((?:17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q6_K"
--tensor-type "(blk|BLK)\.(27|31)\.attn_output=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.ssm_alpha=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.attn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(28|30)\.ssm_out=Q6_K"
--tensor-type "(blk|BLK)\.((?:17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(11|15|19)\.attn_k=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.ssm_beta=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.attn_qkv=Q6_K"
--tensor-type "(blk|BLK)\.(11|15|19)\.attn_q=Q6_K"
--tensor-type "(blk|BLK)\.(11|15|19)\.attn_v=Q6_K"
--tensor-type "(blk|BLK)\.((?:26|27|28|29|30))\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25))\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13|14|15|16))\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13|14|15|16))\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.(22|(?:24|25|26)|29)\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.(19|23)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.([0-3])\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.(3|7|11|15)\.attn_output=Q4_K"
--tensor-type "(blk|BLK)\.([0-3])\.ffn_gate=Q4_K"
--tensor-type ".*output_norm.*=F16"

</details>

Quants — Ornith-1.0-9B (previous generation)

Built with an older ASHQ1 revision (only nextn.* pinned as MTP, output/embd at IQ4_NL). The v7 configs above are not bit-identical to these files. Non-MTP 1.0 quants (deepreinforce-ai_Ornith-1.0-9B-BF16-ASHQ1-*) in commit history.

Hardware Requirements

Notes

• Source BF16 (1.5): MTP-ready BF16 with the head merged in (blk.32 + nextn.*)
• Imatrix: custom Ornith-1.5 imatrix (248 entries)
• Architecture: Qwen3.5-9B (32 layers, GQA, SSM, MTP, 262k context)
• MTP heads deployed at Q8_0, output/token_embd at Q5_K
• Quantization and tuning by wepiqx
• Built with llama.cpp
• License: MIT