Downloads · 30 days
93
16% of all-time downloads
peasantsmith/gpt-oss-20b-PS-IQ4_NL-GGUF
gpt-oss-20b-PS-IQ4_NL-GGUF is a machine learning model from peasantsmith. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
A quality-focused GGUF requant of openai/gpt-oss-20b, built entirely from the original OpenAI weights — no third-party quant as source, no MXFP4 shortcut beyond what OpenAI itself ships.
Downloads · 30 days
93
16% of all-time downloads
All-time downloads
570
Public
Repo size
13.5 GB
Likes
1
Public
Click a slice to open those files.
.gguf13.5 GB · 100%
From the Hugging Face model README
A quality-focused GGUF requant of openai/gpt-oss-20b, built entirely from the original OpenAI weights — no third-party quant as source, no MXFP4 shortcut beyond what OpenAI itself ships.
This model is geometrically incapable of IQ2_XXS/Q2_K/Q6_K: every quantizable tensor is 2880 columns wide (n_embd 2880 % 256 = 64), and all 256-element-block quant formats hard-require cols % 256 == 0. The 32-element-block formats (IQ4_NL, Q5_0, Q8_0) are the highest-fidelity floor this architecture permits. This is the same finding documented for gpt-oss-120b; published "Q2_K" files for gpt-oss models by third parties are silently-fallen-back 4-bit-size files.
openai/gpt-oss-20b official safetensors (experts natively MXFP4 — the only form in which they exist anywhere)convert_hf_to_gguf.py (experts preserved in original MXFP4, everything else BF16→F16)maxdiff 0.000e+00 on independent reference decode; e8m0 scale range 117–129, max |value| ≈ 24)llama-quantize tensor-split requant (below)No fidelity was lost decoding: the decode reproduces the original MXFP4 weight values exactly; the only quantization step is the final split.
| Class | Tensors | Type |
|---|---|---|
| Expert gate/up weights | 48 | IQ4_NL |
| Expert down weights | 24 | Q5_0 |
| Attention, router, output, shared projections | 98 | Q8_0 |
| Norms and biases | 289 | F32 (kept) |
Down-projection errors land directly on the residual stream, so down gets one full step above gate/up. Attention and output stay at Q8_0. Imatrix is not consumed by 32-block types, so none was used.
File: gpt-oss-20b-PS-IQ4_NL.gguf — 13.50 GB (≈5.2 bits/param vs upstream 20.9B total params)
SHA-256: 22f66ec4fd0b6a7e69c490456234e132775b2c210926dc124cd46b7d8b560c89
| Test | Result |
|---|---|
| Greedy smoke ("The capital of France is") | ✅ correct, with clean reasoning block |
| LRU cache code task (8k ctx, greedy) | ✅ correct OrderedDict implementation, O(1) get/put, eviction |
| Needle recall ("9137 steps", 8k ctx) | ✅ exact answer |
| Perplexity | not measured — gpt-oss is harmony-format-only; raw-text PPL is invalid on this family (the F16 itself produces non-finite loss on wikitext) |
-ngl 99 -ts 1,1, ctx 8192, greedy)| Metric | Value |
|---|---|
| Prompt processing | 229–389 t/s |
| Generation | 86–91 t/s |
Fully VRAM-resident on 2× 12 GB. For 12 GB single-GPU machines, offload experts: llama-server -m ... -ngl 99 --n-cpu-moe 16 (documented fallback; not the tested config).
Harmony chat format is required (use the jinja chat template shipped in this repo's tokenizer files / llama.cpp built-in). No MTP head exists for this architecture.
Upstream model by OpenAI (Apache-2.0). Requant by Peasant Smith.