Downloads · 30 days
169
9% of all-time downloads
Ornith-1.0-9B-ASHQ1-MTP-GGUF
wepiqx/Ornith-1.0-9B-ASHQ1-MTP-GGUF
Ornith-1.0-9B-ASHQ1-MTP-GGUF is a text generation model from wepiqx. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as mit.
ASHQ1 quantization of Ornith-1.0-9B with explicit MTP (Multi-Token Prediction) handling — MTP heads are kept at Q80 and excluded from the classifier budget, preserving their quality while the priority queue optimises…
All-time downloads
1.8K
Public
Repo size
25.4 GB
Likes
4
Public
Repo makeup
Click a slice to open those files.
.gguf25.4 GB · 100%
At a glance
- Task
- Text Generation
- Library
- gguf
- License
- mit
- Access
- Public
- Created
- Jul 1, 2026
- Updated
- Jul 1, 2026
- SHA
- 529578b7
Try a prompt
Base models
Model card
From the Hugging Face model README
Ornith-1.0-9B — ASHQ1 Quantization (MTP-Aware)
ASHQ1 quantization of Ornith-1.0-9B with explicit MTP (Multi-Token Prediction) handling — MTP heads are kept at Q8_0 and excluded from the classifier budget, preserving their quality while the priority queue optimises the main transformer layers.
Huge thanks to protoLabsAI/Ornith-1.0-9B-MTP-GGUF for providing the BF16 source with MTP heads — we adapted it with ASHQ1 to deliver both speed and quality.
Note: File names contain "BF16" for HuggingFace parser compatibility — these are ASHQ1 quants, not BF16.
Quants
| File | Size | Method | Description |
|---|---|---|---|
Ornith-1.0-9B-MTP-BF16-ASHQ1-4850.gguf | 4.7 GB | ASHQ1 (v6, --allow-q3-or-lower) | Tiny — uses IQ tiers to fit 4.7 GB, MTP heads at Q8_0 |
Ornith-1.0-9B-MTP-BF16-ASHQ1-5500.gguf | 5.4 GB | ASHQ1 (v6) | Compact — beats i1-Q6_K (7.0 GB), 23% smaller |
Ornith-1.0-9B-MTP-BF16-ASHQ1-6500.gguf | 6.4 GB | ASHQ1 (v6) | Balanced — near i1-Q8_0 quality at 33% smaller |
Ornith-1.0-9B-MTP-BF16-ASHQ1-7300.gguf | 7.2 GB | ASHQ1 (v6) | Best — surpasses i1-Q8_0 quality, 25% smaller |
All comparisons are against i1 quants (uniform quantization with bartowski's Ornith imatrix). ASHQ1 consistently beats them at smaller sizes: the priority queue allocates bits where they matter most. See wepiqx/ASHQ1 for full benchmarks across architectures. MTP heads are deployed at Q8_0 and excluded from the classifier budget.
Note: The 4850 variant uses IQ tiers via
--allow-q3-or-lowerto fit the tight budget — quality is roughly on par with Q4_K_S, while 5500+ delivers a significant step up. It's best suited for lighter tasks, while coding and agentic workloads may prefer 5500+. Feel free to try it either way.
How ASHQ1 Works
See wepiqx/ASHQ1 for the full method, source code, and benchmarks.
Usage
llama.cpp
# Chat mode with built-in Jinja template (recommended)
llama-cli \
-m Ornith-1.0-9B-MTP-BF16-ASHQ1-6500.gguf \
--jinja \
-ngl 99 \
-c 8192
Recommended sampling: temperature 0.6–1.0, top_p 0.95, top_k 20.
Ollama
The model's built-in Jinja template is auto-detected by Ollama — no need to specify TEMPLATE manually.
- Create a
Modelfile:
FROM ./Ornith-1.0-9B-MTP-BF16-ASHQ1-6500.gguf
PARAMETER num_ctx 8192
PARAMETER temperature 0.6
PARAMETER top_k 20
PARAMETER top_p 0.95
- Build and run:
ollama create ornith-ashq1-mtp-6500 -f Modelfile
ollama run ornith-ashq1-mtp-6500
LM Studio
- Open LM Studio
- Drag the desired GGUF file (4850, 5500, 6500, or 7300) into the app
- Set GPU Offload to 99 layers
- Set context length to 8192+
- Set preset sampling:
temperature 0.6,top_p 0.95,top_k 20 - Start chatting
Quantization Configs
Generated by ASHQ1 priority queue (v6) — files in this repo were produced with these exact llama-quantize arguments.
4850 Config (--allow-q3-or-lower)
--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(32)\.nextn=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=IQ4_XS"
--tensor-type "(blk|BLK)\.(30)\.ffn_down=IQ3_S"
--tensor-type "(blk|BLK)\.(32)\.attn_v=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.attn_q=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.attn_k=Q4_K"
--tensor-type "(blk|BLK)\.((?:26|27|28|29))\.ffn_down=IQ3_XXS"
--tensor-type "(blk|BLK)\.((?:22|23|24|25))\.ffn_down=IQ2_S"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21))\.ffn_down=IQ2_XXS"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q=Q5_K"
--tensor-type "(blk|BLK)\.(0|6|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q5_K"
--tensor-type "(blk|BLK)\.(0|6|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q5_K"
--tensor-type "(blk|BLK)\.(0|6|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q5_K"
--tensor-type "(blk|BLK)\.(0|6|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q5_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k=Q5_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_output=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-5])\.attn_gate=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-5])\.ssm_beta=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-5])\.ssm_alpha=Q4_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-5])\.attn_qkv=Q4_K"
--tensor-type "(blk|BLK)\.(28|30)\.ssm_out=IQ4_XS"
--tensor-type "(blk|BLK)\.((?:24|25)|29)\.ssm_out=IQ3_S"
--tensor-type "(blk|BLK)\.((?:21|22)|26)\.ssm_out=IQ3_XXS"
--tensor-type "(blk|BLK)\.(18|20)\.ssm_out=IQ2_S"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=IQ2_XXS"
--tensor-type ".*output_norm.*=F16"
5500 Config (base=Q5_K_M)
--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(32)\.nextn=Q8_0"
--tensor-type "(blk|BLK)\.(3)\.attn_q=Q6_K"
--tensor-type "(blk|BLK)\.(3)\.attn_k=Q6_K"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.((?:20|21)|(?:24|25)|28|30)\.attn_qkv=Q6_K"
--tensor-type "(blk|BLK)\.(3|32)\.attn_v=Q6_K"
--tensor-type "(blk|BLK)\.((?:20|21)|(?:24|25)|28|30)\.ssm_beta=Q6_K"
--tensor-type "(blk|BLK)\.((?:20|21)|(?:24|25)|28|30)\.ssm_alpha=Q6_K"
--tensor-type "(blk|BLK)\.((?:20|21)|(?:24|25)|28|30)\.attn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(31)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.(30)\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.(1)\.attn_gate=Q4_K"
--tensor-type "(blk|BLK)\.(1)\.ssm_alpha=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.attn_q=Q4_K"
--tensor-type "(blk|BLK)\.(1)\.ssm_beta=Q4_K"
--tensor-type "(blk|BLK)\.(1)\.attn_qkv=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30)|32)\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.((?:26|27|28|29|30|31))\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.(0|2|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|22|26|29)\.attn_qkv=Q5_K"
--tensor-type "(blk|BLK)\.(0|2|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|22|26|29)\.ssm_alpha=Q5_K"
--tensor-type "(blk|BLK)\.(7|11|15|19|23|27|(?:31|32))\.attn_k=Q5_K"
--tensor-type "(blk|BLK)\.(7|11|15|19|23|27|31)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(0|2|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|22|26|29)\.attn_gate=Q5_K"
--tensor-type "(blk|BLK)\.(7|11|15|19|23|27|31)\.attn_q=Q5_K"
--tensor-type "(blk|BLK)\.(0|2|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|22|26|29)\.ssm_beta=Q5_K"
--tensor-type "(blk|BLK)\.((?:26|27|28|29|30|31))\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25)|32)\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25)|32)\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29))\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|32)\.attn_output=Q4_K"
--tensor-type ".*output_norm.*=F16"
6500 Config (base=Q5_K_M)
--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(32)\.nextn=Q8_0"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q6_K"
--tensor-type "(blk|BLK)\.(0|(?:9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|15|19|23|27|31)\.attn_v=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q8_0"
--tensor-type "(blk|BLK)\.(0|(?:9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|15|19|23|27|31)\.attn_k=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|15|19|23|27|31)\.attn_q=Q8_0"
--tensor-type "(blk|BLK)\.(11)\.attn_q=Q6_K"
--tensor-type "(blk|BLK)\.(11)\.attn_k=Q6_K"
--tensor-type "(blk|BLK)\.(11)\.attn_v=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|8)\.ssm_beta=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|8)\.attn_qkv=Q6_K"
--tensor-type "(blk|BLK)\.((?:14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|8)\.ssm_alpha=Q6_K"
--tensor-type "(blk|BLK)\.((?:28|29|30))\.ssm_out=Q6_K"
--tensor-type "(blk|BLK)\.((?:14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q6_K"
--tensor-type "(blk|BLK)\.([1-2]|[4-6]|8)\.attn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(27|31)\.attn_output=Q6_K"
--tensor-type "(blk|BLK)\.(32)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(32)\.attn_k=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.attn_q=Q4_K"
--tensor-type "(blk|BLK)\.((?:26|27|28|29|30))\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25)|32)\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.(18|(?:20|21|22)|(?:24|25|26))\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13))\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13))\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.(19|23)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.([0-3]|32)\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.([0-3]|32)\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.(3|7|11|15|32)\.attn_output=Q4_K"
--tensor-type ".*output_norm.*=F16"
7300 Config (base=Q5_K_M)
--output-tensor-type Q5_K
--token-embedding-type Q5_K
--tensor-type "(blk|BLK)\.(32)\.nextn=Q8_0"
--tensor-type "(blk|BLK)\.(31)\.ffn_down=Q8_0"
--tensor-type "(blk|BLK)\.(27|31)\.attn_output=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_v=Q8_0"
--tensor-type "(blk|BLK)\.(28|30)\.ssm_out=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q8_0"
--tensor-type "(blk|BLK)\.((?:16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q8_0"
--tensor-type "(blk|BLK)\.((?:16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q8_0"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q=Q8_0"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k=Q8_0"
--tensor-type "(blk|BLK)\.(32)\.attn_k=Q6_K"
--tensor-type "(blk|BLK)\.((?:22|23|24|25|26|27|28|29|30))\.ffn_down=Q6_K"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13|14|15))\.ffn_up=Q6_K"
--tensor-type "(blk|BLK)\.((?:4|5|6|7|8|9|10|11|12|13|14|15))\.ffn_gate=Q6_K"
--tensor-type "(blk|BLK)\.(18|(?:20|21|22)|(?:24|25|26)|29)\.ssm_out=Q6_K"
--tensor-type "(blk|BLK)\.(19|23)\.attn_output=Q6_K"
--tensor-type "(blk|BLK)\.(32)\.attn_v=Q5_K"
--tensor-type "(blk|BLK)\.(32)\.ffn_gate=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.ffn_up=Q4_K"
--tensor-type "(blk|BLK)\.(32)\.attn_q=Q4_K"
--tensor-type "(blk|BLK)\.((?:18|19|20|21))\.ffn_down=Q5_K"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17)|32)\.ffn_down=Q4_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.attn_norm=F16"
--tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32))\.post_attention_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_k_norm=F16"
--tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|(?:31|32))\.attn_q_norm=F16"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16"
--tensor-type "(blk|BLK)\.([0-3])\.ffn_gate=Q5_K"
--tensor-type "(blk|BLK)\.([0-3])\.ffn_up=Q5_K"
--tensor-type "(blk|BLK)\.((?:9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=Q5_K"
--tensor-type "(blk|BLK)\.(7|11|15)\.attn_output=Q5_K"
--tensor-type "(blk|BLK)\.([0-2]|[4-6]|8)\.ssm_out=Q4_K"
--tensor-type "(blk|BLK)\.(3|32)\.attn_output=Q4_K"
--tensor-type ".*output_norm.*=F16"
Hardware Requirements
| Variant | Size (GGUF) | VRAM (ctx 8192) | Minimum GPU |
|---|---|---|---|
| 4850 | 4.7 GB | ~5.8 GB | 8 GB |
| 5500 | 5.4 GB | ~6.5 GB | 8 GB |
| 6500 | 6.4 GB | ~7.5 GB | 8 GB |
| 7300 | 7.2 GB | ~8.3 GB | 12 GB |
Notes
- Source BF16: protoLabsAI/Ornith-1.0-9B-MTP-GGUF — huge thanks to protoLabsAI for the MTP-ready BF16
- Imatrix: bartowski's Ornith imatrix (248 entries, 6 datasets)
- Architecture: Qwen3.5-9B (32 layers, GQA, SSM, MTP, 262k context)
- MTP heads deployed at Q8_0
- License: MIT
- Quantization and tuning by wepiqx
- Built with llama.cpp
- Task
- Text Generation
- Library
- gguf
- License
- mit
- Languages
- en
- Created
- Jul 1, 2026
- Updated
- Jul 1, 2026