Downloads · 30 days
753
26% of all-time downloads
zSaiko/Pantheon-Reasoning-26B-A4B-1.1-V2-APEX-GGUF
Pantheon-Reasoning-26B-A4B-1.1-V2-APEX-GGUF is a text generation model from zSaiko. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
Mixed-precision APEX GGUF quantizations of Gryphe/Pantheon-Reasoning-26B-A4B-1.1-V2.
Downloads · 30 days
753
26% of all-time downloads
All-time downloads
2.9K
Public
Repo size
184 GB
Likes
1
Public
Click a slice to open those files.
.gguf52.4 GB · 100%
From the Hugging Face model README
Mixed-precision APEX GGUF quantizations of Gryphe/Pantheon-Reasoning-26B-A4B-1.1-V2.
These quants use a Pantheon-specific importance matrix and per-tensor mixed precision rather than a uniform GGUF quantization scheme.
SORRY, Im pretty sure this quantization is somewhat shit after some personal use? If not just comment BUT I basically went off and expected a quantization recipe of StyleTuneV2 to work well here but that model only had lm_head trained and this is vastly differently trained it seems so that was pretty stupid :P - sorry, my first quant
| Variant | Description |
|---|---|
| APEX iQuality | Higher-fidelity mixed-precision quant. Approximately 5.89 BPW. |
| APEX iQuality — Q8 LM Head | Same iQuality tensor allocation, except output.weight is stored at Q8_0 for testing LM-head precision. |
| APEX iCompact | Smaller mixed-precision variant at approximately 4.31 BPW. |
Because APEX uses per-tensor mixed precision, the standard quant type at the end of each filename is an approximate size/BPW class rather than a description of every tensor in the file.
Measured against BF16 reference logits on WikiText-2 using llama.cpp, 128 × 512-token chunks (32,640 scored next-token positions):
| Variant | Mean KLD | Same top token | RMS Δp |
|---|---|---|---|
| APEX iQuality | 0.861731 | 72.387% | 8.638% |
| APEX iQuality — Q8 LM Head | 0.856834 | 72.540% | 8.636% |
| APEX iCompact | 1.386163 | 61.759% | 11.638% |
The Q8 LM-head experiment produced only a small aggregate improvement over the standard iQuality variant.
Gemma 4 contains tensor dimensions that are incompatible with some requested K-quant formats. In particular, ffn_down_exps.weight has a 704-wide dimension, so llama.cpp may use compatible fallback formats when necessary.
Observed examples include:
IQ4_XS → stored as IQ4_NLQ5_K → stored as Q5_1Q6_K → stored as Q8_0These are expected shape-dependent fallbacks.
See the original model card for architecture, training details, intended usage, and model-specific notes:
Gryphe/Pantheon-Reasoning-26B-A4B-1.1-V2
Original model by Gryphe.
APEX mixed-precision methodology/configuration derived from the work by EmanuelOverride on Gemma 4 APEX GGUF quantization.