Downloads · 30 days
28
7% of all-time downloads
amanwalksdownthestreet/gpt-oss-120b-exl3
gpt-oss-120b-exl3 is a text generation model from amanwalksdownthestreet. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Quantized with ExLlamaV3 1.0.0 from openai/gpt-oss-120b. Opt quants are per-tensor mixes of the six base quants, selected by measured KLD — measurement.json included. Additional quantizations can be generated by anyon…
Downloads · 30 days
28
7% of all-time downloads
All-time downloads
409
Public
Repo size
718 GB
Likes
2
Public
Click a slice to open those files.
.json964 KB · 99%
From the Hugging Face model README
Quantized with ExLlamaV3 1.0.0 from openai/gpt-oss-120b. Opt quants are per-tensor mixes of the six base quants, selected by measured KLD — measurement.json included. Additional quantizations can be generated by anyone using the measure.py output I recorded and the base quants.
| Quant | Size | AIME24 | AIME25 | GPQA-D |
|---|---|---|---|---|
openai/gpt-oss-120b — published | 61 GB | 95.8 | 92.5 | 80.1 |
| 4.8bpw-opt-h16 | 70 GB | 90.0 <sub>−5.8</sub> | 86.7 <sub>−5.8</sub> | 73.7 <sub>−6.4</sub> |
| 4.3bpw-opt-h16 | 63 GB | 93.3 <sub>−2.5</sub> | 86.7 <sub>−5.8</sub> | 74.7 <sub>−5.4</sub> |
| 3.7bpw-opt-h16 | 55 GB | 86.7 <sub>−9.1</sub> | 80.0 <sub>−12.5</sub> | 74.7 <sub>−5.4</sub> |
| 2.6bpw-opt-h16 | 39 GB | 83.3 <sub>−12.5</sub> | 73.3 <sub>−19.2</sub> | 71.7 <sub>−8.4</sub> |
| 2.3bpw-opt-h16 | 35 GB | 83.3 <sub>−12.5</sub> | 70.0 <sub>−22.5</sub> | 71.7 <sub>−8.4</sub> |
<sub>Δ = vs published row.</sub>
Don't use the branches at or above 4 bpw. gpt-oss-120b ships natively in MXFP4 — the MoE experts are already 4-bit (~4.25 bpw with block scales), only attention/embeddings/lm_head are BF16, and the full checkpoint is 61 GB. Quantizing above that adds size without adding information: extra bits re-encode already-4-bit values more faithfully, they don't recover precision the source never had. The 4.3 and 4.8 branches are larger than the original and show no quality advantage over it. Use the original. The sub-4 bpw branches are the ones that do something the original can't.
<sub>lm-eval 0.4.11 → TabbyAPI / ExLlamaV3 v1.1.0, harmony chat template, reasoning_effort=high, 122,880-token budget, temperature 0.</sub>
| Branch | Bitrate | Size |
|---|---|---|
4.8bpw-opt-h16 | 4.8 bpw | 70 GB |
4.3bpw-opt-h16 | 4.3 bpw | 63 GB |
3.7bpw-opt-h16 | 3.7 bpw | 55 GB |
2.6bpw-opt-h16 | 2.6 bpw | 39 GB |
2.3bpw-opt-h16 | 2.3 bpw | 35 GB |
| Branch | Bitrate | Size |
|---|---|---|
8.0bpw-h16 | 8.0 bpw | 116 GB |
6.0bpw-h16 | 6.0 bpw | 87 GB |
5.0bpw-h16 | 5.0 bpw | 73 GB |
4.0bpw-h16 | 4.0 bpw | 59 GB |
3.0bpw-h16 | 3.0 bpw | 45 GB |
2.0bpw-h16 | 2.0 bpw | 31 GB |