Downloads · 30 days
75
9% of all-time downloads
inkOrCloud/Ornith-1.0-35B-abliterix-UD-like-GGUF
Ornith-1.0-35B-abliterix-UD-like-GGUF is a machine learning model from inkOrCloud. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Quantization Notes (UD-like / Unsloth-layout-reproduced)
Downloads · 30 days
75
9% of all-time downloads
All-time downloads
821
Public
Repo size
692 GB
Likes
0
Public
Click a slice to open those files.
.gguf205 GB · 100%
From the Hugging Face model README
Quantization Notes (UD-like / Unsloth-layout-reproduced)
This repository provides GGUF quantizations of Ornith-1.0-35B-abliterix (the Ornith-1.0-35B model, based on Qwen3.5-35B-A3B, after ablation fine-tuning with abliterix using the vLLM backend / EGA expert-granularity steering).
| File | Precision / Layout | Description |
|---|---|---|
Ornith-1.0-35B-abliterix-BF16-00001-of-00002.gguf / -00002-of-00002.gguf | BF16 (unquantized, sharded) | Full BF16 weights of the ablated model (69.4GB, 2 shards); baseline and highest-quality version |
Ornith-1.0-35B-abliterix-UD-like-Q8_K_XL.gguf | Mixed layout (default Q8_0 + 12×BF16 sensitive tensors + F32 small tensors) | Reproduces the official UD-Q8_K_XL layout |
Ornith-1.0-35B-abliterix-UD-like_Q6_K_XL.gguf | Mixed layout (default Q8_0 + 78×Q6_K MoE tensors + F32 small tensors) | Reproduces the official UD-Q6_K_XL layout |
Ornith-1.0-35B-abliterix-Q8_0.gguf | Q8_0 (standard, uniform) | Plain llama.cpp Q8_0 quantization with official imatrix |
Ornith-1.0-35B-abliterix-Q6_K.gguf | Q6_K (standard, uniform) | Plain llama.cpp Q6_K quantization with official imatrix |
Note: The BF16 file is split into 2 shards because it exceeds Hugging Face's 50GB single-file limit (
llama-gguf-split --split-max-size 40G); all other files are single files under 50GB. Shards must be kept in the same directory (llama.cpp auto-detects the-00001-of-00002naming).
The UD-like-Q8_K_XL and UD-like_Q6_K_XL files are per-tensor quantization layout reproductions (UD-like) — they are NOT official UD-* models generated by Unsloth Studio:
UD-Q8_K_XL / UD-Q6_K_XL GGUFs and imatrix_unsloth.gguf_file published in unsloth/Ornith-1.0-35B-GGUF;convert_hf_to_gguf.py (BF16 conversion) + llama-quantize --imatrix --tensor-type (per-tensor override quantization following the official layout);UD-like / Unsloth-layout-reproduced — explicitly NOT official UD quantization (official UD requires Unsloth Studio's closed-source pipeline + proprietary calibration and cannot be publicly reproduced).Q8_0blk.34.ffn_down_exps.weightblk.38.ffn_down_exps.weightblk.39: attn_q/k/v/output, ffn_gate/up/down_exps, ffn_gate/up/down_shexpssm_alpha / ssm_beta (30 each) kept in F32*_norm.weight, ffn_gate_inp*, ssm_a, ssm_conv1d, ssm_dt.bias) kept in F32 (architecture default)Q8_0ffn_gate_exps.weight / ffn_up_exps.weight of blk.0–blk.38 (78 MoE expert tensors in total) use Q6_Kblk.39's ffn_gate/up_exps are the exception, kept in Q8_0ssm_alpha / ssm_beta kept in F32# 1. HF safetensors -> BF16 GGUF (single 69.4GB file)
python llama.cpp/convert_hf_to_gguf.py models/abliterix \
--outfile Ornith-1.0-35B-abliterix-BF16.gguf --outtype bf16
# 1b. Split BF16 (HF single-file limit is 50GB)
llama-gguf-split --split-max-size 40G \
Ornith-1.0-35B-abliterix-BF16.gguf \
Ornith-1.0-35B-abliterix-BF16-split.gguf
# 2a. UD-like-Q8_K_XL (--tensor-type-file holds the per-tensor override rules; see layout above)
llama-quantize --imatrix imatrix_unsloth.gguf_file \
--tensor-type-file overrides-q8kxl.txt \
Ornith-1.0-35B-abliterix-BF16.gguf \
Ornith-1.0-35B-abliterix-UD-like-Q8_K_XL.gguf Q8_0
# 2b. UD-like_Q6_K_XL
llama-quantize --imatrix imatrix_unsloth.gguf_file \
--tensor-type-file overrides-q6kxl.txt \
Ornith-1.0-35B-abliterix-BF16.gguf \
Ornith-1.0-35B-abliterix-UD-like_Q6_K_XL.gguf Q8_0
# 3a. Standard Q8_0
llama-quantize --imatrix imatrix_unsloth.gguf_file \
Ornith-1.0-35B-abliterix-BF16.gguf \
Ornith-1.0-35B-abliterix-Q8_0.gguf Q8_0
# 3b. Standard Q6_K
llama-quantize --imatrix imatrix_unsloth.gguf_file \
Ornith-1.0-35B-abliterix-BF16.gguf \
Ornith-1.0-35B-abliterix-Q6_K.gguf Q6_K
The imatrix comes from the official repository unsloth/Ornith-1.0-35B-GGUF (imatrix_unsloth.gguf_file) and is used to weight quantization error for the MoE expert tensors.
UD-like models reproduce the official quantization layout (per-tensor storage types) only; they do not guarantee byte-level identity, task accuracy, or internal decision pipeline equivalence with official UD models;UD-Q8_K_XL / UD-Q6_K_XL are generated by Unsloth Studio (closed-source); this repository does not use its proprietary calibration process;