Downloads · 30 days
61
4% of all-time downloads
gkraker04/Ornstein3.6-27B-MTP-NSC-ACE-SABER-turbo4-GGUF
Ornstein3.6-27B-MTP-NSC-ACE-SABER-turbo4-GGUF is a image-text-to-text model from gkraker04. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
Downloads · 30 days
61
4% of all-time downloads
All-time downloads
1.7K
Public
Repo size
32 GB
Likes
0
Public
Click a slice to open those files.
.gguf32 GB · 100%
From the Hugging Face model README

This GGUF release introduces turbo4 weight quantization — a custom 4-bit quantization scheme for Ornstein3.6-27B-MTP-NSC-ACE-SABER using buun-llama-cpp's turbo types.
GGML_TYPE_TURBO4_0 tensor typebuild_turbo4.bat (MSVC + CUDA 12.8 + Ninja, SM_86)vec_dot tuning yet.Use this for:
Use Q4_K_M/Q6_K for production inference until turbo4 performance improves.
These GGUF files are quantized from GestaltLabs/Ornstein3.6-27B-MTP-NSC-ACE-SABER, a staged NSC-ACE -> SABER -> Ornstein checkpoint.
| Stage | Purpose | Key settings |
|---|---|---|
| NSC-ACE | Internally steered rollout committee with reward on convergent tool-call structure. | agentic tool-call convergence |
| SABER | Compliance-first calibration, then KLD/PPL selection. | 1046 HarmBench + AdvBench-style prompts; selected row reports 968/1046 by keyword proxy |
| Ornstein SFT | Reasoning/personality refinement after SABER. | premium reasoning v1, 1 epoch, 50 steps, rank 32/32, dropout 0.05, cosine schedule |
| GGUF conversion | MTP-capable llama.cpp conversion and quantization. | llama.cpp PR #22673, commit e7b484815 |
| Metric | Value |
|---|---|
| SABER selected compliance proxy | 92.54% (968/1046 eval prompts) |
| SABER selected keyword residual | 7.46% (78/1046 eval prompts) |
| SABER selected HarmBench classifier ASR | 0.67% (7/1046 eval prompts) |
| SABER selected KLD | 0.008302 |
| SABER selected PPL ratio | 1.103853 |
| SABER selected post PPL | 17.5988 |
| SABER selected base PPL | 15.9431 |
| MTP status | present and verified |
mtp_num_hidden_layers | 1 |
Source mtp.* tensors | 15 |
| Corrected source tensor count | 866 |
This repository hosts llama.cpp/GGUF builds for GestaltLabs/Ornstein3.6-27B-MTP-NSC-ACE-SABER. GGUF artifacts live here so the main safetensors repository stays focused on the source checkpoint.
These files are built from an MTP-capable source checkpoint:
| MTP check | Value |
|---|---|
mtp_num_hidden_layers | 1 |
mtp_use_dedicated_embeddings | false |
Source mtp.* tensors | 15 |
| Corrected source tensor count | 866 |
| Conversion path | llama.cpp PR #22673, commit e7b484815 |
MTP support is present and verified. This release includes mtp_num_hidden_layers=1 and MTP/nextn tensors in the GGUF validation path. Use a llama.cpp build with Qwen3.5/Qwen3.6 MTP support and run with --spec-type mtp.
This repo includes mmproj-Ornstein3.6-27B-MTP-NSC-ACE-SABER-F16.gguf, converted from the original multimodal Ornstein 27B vision encoder. Use it with llama.cpp's multimodal path alongside any of the text GGUF quants.
| File | Status | Notes |
|---|---|---|
mmproj-Ornstein3.6-27B-MTP-NSC-ACE-SABER-F16.gguf | uploaded | Vision encoder / projector for image input |
Ornstein3.6-27B-MTP-NSC-ACE-SABER-F16-MTP.gguf | uploaded | Full GGUF conversion source / highest local fidelity |
Ornstein3.6-27B-MTP-NSC-ACE-SABER-Q8_0-MTP.gguf | uploaded | Near-full quality, large local file |
Ornstein3.6-27B-MTP-NSC-ACE-SABER-Q6_K-MTP.gguf | uploaded | High-quality local default if memory allows |
Ornstein3.6-27B-MTP-NSC-ACE-SABER-Q5_K_M-MTP.gguf | uploaded | Strong quality/size balance |
Ornstein3.6-27B-MTP-NSC-ACE-SABER-Q5_K_S-MTP.gguf | uploaded | Smaller Q5 option |
Ornstein3.6-27B-MTP-NSC-ACE-SABER-Q4_K_M-MTP.gguf | uploaded | Common balanced local target |
Ornstein3.6-27B-MTP-NSC-ACE-SABER-Q4_K_S-MTP.gguf | uploaded | Smaller Q4 option |
Ornstein3.6-27B-MTP-NSC-ACE-SABER-Q3_K_L-MTP.gguf | uploaded | Lower-memory Q3 option |
Ornstein3.6-27B-MTP-NSC-ACE-SABER-Q3_K_M-MTP.gguf | uploaded | Smaller Q3 balance |
Ornstein3.6-27B-MTP-NSC-ACE-SABER-Q3_K_S-MTP.gguf | uploaded | Small Q3 option |
Ornstein3.6-27B-MTP-NSC-ACE-SABER-Q2_K-MTP.gguf | uploaded | Minimum-size target; quality loss expected |
Ornstein3.6-27B-MTP-NSC-ACE-SABER-turbo4-MTP-v3.gguf | uploaded | |
Ornstein3.6-27B-MTP-NSC-ACE-SABER-turbo4-MTP-v4-i3.gguf | uploaded |
All listed GGUF artifacts have been uploaded.
| Quant | Best fit |
|---|---|
| F16 | Maximum fidelity when disk/RAM are not a concern |
| Q8_0 | Very high fidelity local inference |
| Q6_K | Recommended high-quality local starting point |
| Q5_K_M | Strong balance for quality and size |
| Q4_K_M | Practical default for constrained machines |
| Q3_K_M / Q3_K_S | Low-memory experiments |
| Q2_K | Smallest target; use only when memory is the hard constraint |
For agentic/tool-calling workloads, prefer Q6_K, Q5_K_M, or Q4_K_M when possible. Very low quants can shift structured output before they obviously degrade prose.
llama-server \
-m Ornstein3.6-27B-MTP-NSC-ACE-SABER-Q6_K-MTP.gguf \
--mmproj mmproj-Ornstein3.6-27B-MTP-NSC-ACE-SABER-F16.gguf \
--spec-type mtp \
-ngl 999 \
-c 32768
Use a llama.cpp build that includes Qwen3.5/Qwen3.6 MTP support.
GestaltLabs/Ornstein3.6-27B-MTP-NSC-ACE-SABERFor technical details on the turbo4 weight quantization implementation, see buun-llama-cpp PR #61.
Support this work on Ko-fi.