Downloads · 30 days
1.1K
1% of all-time downloads
vcruz305/Ornith-1.0-35B-AEON-Ultimate-Uncensored-GGUF
Ornith-1.0-35B-AEON-Ultimate-Uncensored-GGUF is a text generation model from vcruz305. Use it when you need the model to write or continue text. The card lists the license as other.
GGUF quantizations of AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-BF16, produced from the BF16 source weights using importance-matrix calibration.
Downloads · 30 days
1.1K
1% of all-time downloads
All-time downloads
80.1K
Public
Repo size
372 GB
Likes
17
Public
Click a slice to open those files.
.gguf372 GB · 100%
From the Hugging Face model README
GGUF quantizations of AEON-7/Ornith-1.0-35B-AEON-Ultimate-Uncensored-BF16, produced from the BF16 source weights using importance-matrix calibration.
Two variants are provided: standard trunk (no MTP) and MTP-grafted (with Multi-Token Prediction block for speculative decoding).
For standard autoregressive inference. Smaller files, no speculative decoding overhead.
| File | Quant | Size | BPW | Target GPU |
|---|---|---|---|---|
ornith-aeon-35b-Q8_0.gguf | Q8_0 | 35 GB | ~8.5 | 48GB+ (near-lossless) |
ornith-aeon-35b-Q6_K.gguf | Q6_K | 27 GB | ~6.6 | 32GB+ (high quality) |
ornith-aeon-35b-Q5_K_M.gguf | Q5_K_M | 24 GB | ~5.7 | 24GB (quality-first) |
ornith-aeon-35b-Q4_K_M.gguf | Q4_K_M | 21.2 GB | ~4.8 | 24GB (balanced) |
ornith-aeon-35b-Q4_K_S.gguf | Q4_K_S | 19.9 GB | ~4.6 | 24GB fallback |
ornith-aeon-35b-IQ4_XS.gguf | IQ4_XS | 18.7 GB | ~4.3 | 20GB (RTX 4000 Ada) |
ornith-aeon-35b-Q3_K_M.gguf | Q3_K_M | 16.8 GB | ~3.9 | 16GB GPUs |
ornith-aeon-35b-Q2_K.gguf | Q2_K | 12.9 GB | ~3.0 | 12GB GPUs |
ornith-aeon-35b-IQ1_M.gguf | IQ1_M | 8.2 GB | ~1.8 | 8GB GPUs |
ornith-aeon-35b.imatrix | - | 184 MB | - | Importance matrix |
These GGUFs contain all 785 MTP tensors grafted from the base Qwen/Qwen3.5-35B-A3B model, adding a full MTP prediction block (blk.40) with 256 MoE experts. Use with --spec-type draft-mtp for speculative decoding. MTP is bundled in the GGUF -- no separate draft file needed.
| File | Quant | Size | BPW | Target GPU |
|---|---|---|---|---|
ornith-aeon-35b-MTP-Q8_0.gguf | Q8_0 | 36 GB | ~8.5 | 48GB+ (near-lossless) |
ornith-aeon-35b-MTP-Q6_K.gguf | Q6_K | 28 GB | ~6.6 | 32GB+ (high quality) |
ornith-aeon-35b-MTP-Q5_K_M.gguf | Q5_K_M | 24 GB | ~5.7 | 24GB (quality-first) |
ornith-aeon-35b-MTP-Q4_K_M.gguf | Q4_K_M | 21.7 GB | ~4.8 | 24GB (balanced) |
ornith-aeon-35b-MTP-Q4_K_S.gguf | Q4_K_S | 20.4 GB | ~4.6 | 24GB fallback |
ornith-aeon-35b-MTP-IQ4_XS.gguf | IQ4_XS | 19.2 GB | ~4.3 | 20-24GB |
ornith-aeon-35b-MTP-Q3_K_M.gguf | Q3_K_M | 17.2 GB | ~3.9 | 16GB GPUs |
ornith-aeon-35b-MTP-Q2_K.gguf | Q2_K | 13.3 GB | ~3.0 | 12GB GPUs |
ornith-aeon-35b-MTP.imatrix | - | 184 MB | - | Importance matrix |
The original AEON fine-tune lost its 785 MTP weight tensors. HuggingFace Transformers' AutoModelForCausalLM silently drops all mtp.* tensors during loading (_keys_to_ignore_on_load_unexpected = [r"^mtp.*"]). The mtp_num_hidden_layers: 1 in config.json is orphaned metadata from the base Qwen3.5-35B-A3B model.
The MTP-grafted variants restore all 785 MTP tensors (~488 MB BF16) by copying them from the base Qwen3.5-35B-A3B model. This works because Ornith shares identical architecture, hidden dimensions, expert count, and embedding space with its base model. Community-measured acceptance rates for grafted MTP: 58-100%.
Tested: MTP-Q4_K_M on DGX Spark (GB10)
| Mode | Prompt | Generation |
|---|---|---|
| Standard (no MTP) | 47.4 t/s | 34.9 t/s |
Embedded --spec-type draft-mtp | 46.2 t/s | 62.8 t/s |
1.8x generation speedup with the bundled MTP head on a single GPU.
An important distinction for MoE speculative decoding:
--spec-type draft-mtp): Net positive. The MTP head shares the model's KV cache and embedding space. Measured +27-80% generation speedup depending on workload and hardware.-md with a standalone GGUF): Net negative for MoE. A separate draft model triggers expert-union overhead during batch verification -- more expert weight blocks must be read from VRAM, exceeding the savings. Benchmarks show -18% to -52% regression.The MTP-grafted GGUFs in this repo use the embedded approach.
llama-server \
-m ornith-aeon-35b-Q4_K_M.gguf \
-a ornith-aeon-35b \
--host 0.0.0.0 --port 8083 \
-ngl 99 \
--n-cpu-moe 3 \
-fa on \
-ctk q4_0 -ctv q4_0 \
-c 32768 \
-b 2048 -ub 768 \
-np 1 -cb \
-n 16384 \
--temp 0.6 --top-k 20 --top-p 0.95 \
--repeat-penalty 1.1 \
--jinja \
--reasoning-format deepseek \
--reasoning-budget 1024
Requires llama.cpp built from latest master (b9606+). MTP is bundled in the GGUF -- no separate draft file needed.
llama-server \
-m ornith-aeon-35b-MTP-Q4_K_M.gguf \
-a ornith-aeon-35b \
--spec-type draft-mtp \
--host 0.0.0.0 --port 8083 \
-ngl 99 \
--n-cpu-moe 3 \
-fa on \
-ctk q4_0 -ctv q4_0 \
-c 32768 \
-b 2048 -ub 768 \
--parallel 1 \
--temp 0.6 --top-k 20 --top-p 0.95 \
--repeat-penalty 1.1 \
--jinja \
--reasoning-format deepseek \
--reasoning-budget 1024
llama-server \
-m ornith-aeon-35b-MTP-IQ4_XS.gguf \
-a ornith-aeon-35b \
--spec-type draft-mtp \
--host 0.0.0.0 --port 8083 \
-ngl 99 \
--n-cpu-moe 5 \
-fa on \
-ctk q4_0 -ctv q4_0 \
-c 8192 \
-b 1024 -ub 512 \
--parallel 1 \
--temp 0.6 --top-k 20 --top-p 0.95 \
--repeat-penalty 1.1 \
--jinja \
--reasoning-format deepseek \
--reasoning-budget 1024
llama-server \
-m ornith-aeon-35b-MTP-Q6_K.gguf \
-a ornith-aeon-35b \
--spec-type draft-mtp \
--host 0.0.0.0 --port 8083 \
-ngl 99 \
-fa on \
-ctk q8_0 -ctv q8_0 \
-c 32768 \
-b 2048 -ub 768 \
--parallel 1 \
--temp 0.6 --top-k 20 --top-p 0.95 \
--repeat-penalty 1.1 \
--jinja \
--reasoning-format deepseek \
--reasoning-budget 1024
All available in llama.cpp latest master. Work on any CUDA GPU (Ada Lovelace, Ampere, Blackwell):
| Strategy | Flag | Draft Model? | Notes |
|---|---|---|---|
| MTP (embedded) | --spec-type draft-mtp | Bundled in MTP GGUFs | Requires --parallel 1. Measured +80% on Spark. |
| Eagle3 | --spec-type draft-eagle3 | Separate GGUF | PR #18039. No pre-trained drafter for this model yet. |
| DFlash | --spec-type draft-dflash | Separate GGUF | z-lab/Qwen3.5-35B-A3B-DFlash. Best results on vLLM/SGLang at BF16/FP8. |
| N-gram | --spec-default | None | Zero VRAM cost. Marginal benefit on structured output. |
| Parameter | Value | Why |
|---|---|---|
-ngl 99 | Offload all layers to GPU | Maximizes speed |
--spec-type draft-mtp | MTP speculative decoding | Only with MTP-grafted GGUFs |
--parallel 1 | Single slot | Required for MTP |
--n-cpu-moe N | Offload N MoE layers to CPU | Saves ~0.8GB per layer |
-fa on | Flash attention | Reduces KV cache memory |
-ctk q4_0 -ctv q4_0 | Quantize KV cache | Saves ~60% KV cache VRAM |
--jinja | Jinja2 chat template | Required for this model |
--reasoning-format deepseek | Thinking mode | Model uses think tags |
--n-cpu-moe (trades speed for VRAM)-c 8192 or -c 4096-b 512 -ub 256