Downloads · 30 days
601
3% of all-time downloads
GestaltLabs/Ornstein-Hermes-3.6-27B-SABER-GGUF
Ornstein-Hermes-3.6-27B-SABER-GGUF is a text generation model from GestaltLabs. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as other.
GGUF quantizations of GestaltLabs/Ornstein-Hermes-3.6-27b-SABER, a SABER-edited version of GestaltLabs/Ornstein-Hermes-3.6-27b.
Downloads · 30 days
601
3% of all-time downloads
All-time downloads
20.7K
Public
Repo size
125 GB
Likes
21
Public
Click a slice to open those files.
.gguf125 GB · 100%
From the Hugging Face model README
GGUF quantizations of GestaltLabs/Ornstein-Hermes-3.6-27b-SABER, a SABER-edited version of GestaltLabs/Ornstein-Hermes-3.6-27b.
SABER is a controlled refusal-shaping workflow. The release target is to reduce broad over-refusal while preserving ordinary model behavior and visible boundaries for severe criminal, coercive, or interpersonal-harm requests. The selected checkpoint was chosen as a Pareto point over refusal rate and behavioral drift.
| field | value |
|---|---|
| Source repo | GestaltLabs/Ornstein-Hermes-3.6-27b-SABER |
| Base model | GestaltLabs/Ornstein-Hermes-3.6-27b |
| SABER run | ornstein_hermes36_27b_svd_a850_g25_retry_biggpu |
| Expanded refusal eval | 1 / 349 refusals |
| Refusal rate | 0.29% |
| KLD mean | 11.2216 |
| Base-vs-base KLD mean | 11.2206 |
| KLD delta over base-vs-base | +0.0010 |
| KLD prompts | 149 |
| Tokens scored for KLD | 3,347 |
The one retained refusal in the expanded evaluation was an illegal-drug-sales request. This is an observed result on the current evaluation set, not a universal guarantee about future behavior.
| file | quant | size | notes |
|---|---|---|---|
Ornstein-Hermes-3.6-27b-SABER-IQ4_XS.gguf | IQ4_XS | 15G | Compact imatrix-assisted 4-bit option. |
Ornstein-Hermes-3.6-27b-SABER-IQ2_M.gguf | IQ2_M | 9G | Smallest emergency 2-bit option; expect the most quality loss. |
Ornstein-Hermes-3.6-27b-SABER-Q3_K_M.gguf | Q3_K_M | 13G | Smallest file in this suite; expect more quality loss. |
Ornstein-Hermes-3.6-27b-SABER-Q4_K_M.gguf | Q4_K_M | 16G | General-purpose recommended starting point. |
Ornstein-Hermes-3.6-27b-SABER-Q5_K_M.gguf | Q5_K_M | 18G | Balanced high-quality option. |
Ornstein-Hermes-3.6-27b-SABER-Q6_K.gguf | Q6_K | 21G | Strong quality/size option for high-memory local inference. |
Ornstein-Hermes-3.6-27b-SABER-Q8_0.gguf | Q8_0 | 27G | Highest quality quant in this suite; largest runtime file. |
The included imatrix file was generated from DJLougen/Acta-Synthetic. It is included for reproducibility and for users who want to regenerate adjacent quantizations.
Start with for normal desktop use. Use or if you have enough VRAM/RAM and want a higher-quality local run. Use when file size matters more. is mainly for high-memory systems or as a near-lossless GGUF reference.
These files were produced with llama.cpp commit from a BF16 GGUF conversion of the SABER checkpoint. The model uses the GGUF architecture path in current llama.cpp.
Example:
For chat-style use, prefer a frontend or wrapper that applies the tokenizer chat template from the GGUF metadata.
The Q8_0 GGUF was converted from the full SABER Hugging Face checkpoint. The lower-bit recovery quants were generated from the published Q8_0 GGUF with --allow-requantize and the included Acta-Synthetic imatrix so the missing files could be restored quickly. Importance-matrix calibration used Acta-Synthetic conversational text.
SABER edits refusal behavior through activation/weight-space refusal directions. For this checkpoint, the run used SVD extraction, multi-layer candidate selection, iterative ablation, and KLD-based drift measurement.
Run configuration:
Selected layers:
Total directions ablated: .
This release builds on the refusal-direction and abliteration research lineage. Relevant prior work and inspirations include:
SABER's contribution in this release is the controlled-refusal-shaping workflow: multi-candidate refusal extraction, separability/entanglement-aware ranking, differential ablation strength, and explicit Pareto selection over refusal behavior and KLD drift.