Downloads · 30 days
90
9% of all-time downloads
Smoffyy/gpt-oss-20b-instruct-pure-gguf
gpt-oss-20b-instruct-pure-gguf is a text generation model from Smoffyy. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
<img width="900px" src="https://cdn-uploads.huggingface.co/production/uploads/648812cd28c3bccafcd68e4c/-vN17fHowLqWFYqWALTME.png"
Downloads · 30 days
90
9% of all-time downloads
All-time downloads
1K
Public
Repo size
13.8 GB
Likes
1
Public
Click a slice to open those files.
.gguf13.8 GB · 100%
From the Hugging Face model README
This is a quantized version of the official gpt-oss model, created locally using the Official Llama.cpp
It's completely unmodified, no edits, just direct quantizations of the original weights. That's why they're called pure GGUFs.
This is a Mixture-of-Experts (MoE) model: of its 21 billion total parameters, only ~3.6 billion are active per inference step. In practice, this means faster token generation at the cost of higher VRAM usage compared to a dense model of similar active size.
Most quantized models you find online may include minor tweaks that the user may not want. Most of these tweaks, the daily user will not be bother by or even notice. But I find having the purity of a GGUF, knowing it is purely based on the official weights, is better. I'd rather use my own quants that I know are pure.
Straight from the official weights, purely quantized, pure GGUF.
This model is based on gpt-oss, which was released under the Apache 2.0 license. This work and all derivatives are released under Apache 2.0. Should the original model's license change in the future, this version remains under Apache 2.0 in perpetuity.
This model has only one quantization, which is mxfp4 format. Because of this, I have only uploaded one GGUF.
Using this model at a Context Size of 64k, uses around 16GB. This is WITHOUT MoE layers offloaded to CPU.
openai/gpt-oss-20b repository on HuggingFace..git, .gitattributes, LICENSE, README. They're not needed.convert_hf_to_gguf.py from the official llama.cpp source.llama-quantize.Welcome to the gpt-oss series, OpenAI’s open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases.
We’re releasing two flavors of these open models:
gpt-oss-120b — for production, general purpose, high reasoning use cases that fit into a single 80GB GPU (like NVIDIA H100 or AMD MI300X) (117B parameters with 5.1B active parameters)gpt-oss-20b — for lower latency, and local or specialized use cases (21B parameters with 3.6B active parameters)Both models were trained on our harmony response format and should only be used with the harmony format as it will not work correctly otherwise.
[!NOTE] This model card is dedicated to the smaller
gpt-oss-20bmodel. Check outgpt-oss-120bfor the larger model.
gpt-oss-120b run on a single 80GB GPU (like NVIDIA H100 or AMD MI300X) and the gpt-oss-20b model run within 16GB of memory. All evals were performed with the same MXFP4 quantization.You can use gpt-oss-120b and gpt-oss-20b with Transformers. If you use the Transformers chat template, it will automatically apply the harmony response format. If you use model.generate directly, you need to apply the harmony format manually using the chat template or use our openai-harmony package.
To get started, install the necessary dependencies to setup your environment:
pip install -U transformers kernels torch
Once, setup you can proceed to run the model by running the snippet below:
from transformers import pipeline
import torch
model_id = "openai/gpt-oss-20b"
pipe = pipeline(
"text-generation",
model=model_id,
torch_dtype="auto",
device_map="auto",
)
messages = [
{"role": "user", "content": "Explain quantum mechanics clearly and concisely."},
]
outputs = pipe(
messages,
max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])
Alternatively, you can run the model via Transformers Serve to spin up a OpenAI-compatible webserver:
transformers serve
transformers chat localhost:8000 --model-name-or-path openai/gpt-oss-20b
Learn more about how to use gpt-oss with Transformers.
vLLM recommends using uv for Python dependency management. You can use vLLM to spin up an OpenAI-compatible webserver. The following command will automatically download the model and start the server.
uv pip install --pre vllm==0.10.1+gptoss \
--extra-index-url https://wheels.vllm.ai/gpt-oss/ \
--extra-index-url https://download.pytorch.org/whl/nightly/cu128 \
--index-strategy unsafe-best-match
vllm serve openai/gpt-oss-20b
Learn more about how to use gpt-oss with vLLM.
To learn about how to use this model with PyTorch and Triton, check out our reference implementations in the gpt-oss repository.
If you are trying to run gpt-oss on consumer hardware, you can use Ollama by running the following commands after installing Ollama.
# gpt-oss-20b
ollama pull gpt-oss:20b
ollama run gpt-oss:20b
Learn more about how to use gpt-oss with Ollama.
If you are using LM Studio you can use the following commands to download.
# gpt-oss-20b
lms get openai/gpt-oss-20b
Check out our awesome list for a broader collection of gpt-oss resources and inference partners.
You can download the model weights from the Hugging Face Hub directly from Hugging Face CLI:
# gpt-oss-20b
huggingface-cli download openai/gpt-oss-20b --include "original/*" --local-dir gpt-oss-20b/
pip install gpt-oss
python -m gpt_oss.chat model/
You can adjust the reasoning level that suits your task across three levels:
The reasoning level can be set in the system prompts, e.g., "Reasoning: high".
The gpt-oss models are excellent for:
Both gpt-oss models can be fine-tuned for a variety of specialized use cases.
This smaller model gpt-oss-20b can be fine-tuned on consumer hardware, whereas the larger gpt-oss-120b can be fine-tuned on a single H100 node.
@misc{openai2025gptoss120bgptoss20bmodel,
title={gpt-oss-120b & gpt-oss-20b Model Card},
author={OpenAI},
year={2025},
eprint={2508.10925},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2508.10925},
}