gpt-oss-20b-GGUF

GPT-OSS-20B

GPT-OSS-20B is an open-weight reasoning language model developed by OpenAI for advanced reasoning, agentic workflows, and specialized developer applications. It is the smaller model in the GPT-OSS family and is designed for lower-latency inference and local deployment while providing strong capabilities across reasoning, coding, tool use, and structured generation.

The model uses a Mixture-of-Experts (MoE) architecture with approximately 21 billion total parameters and approximately 3.6 billion active parameters per token. It contains 32 experts and activates 4 experts for each token, allowing the model to maintain a large overall parameter capacity while limiting the number of parameters involved in each forward pass.

GPT-OSS-20B supports configurable reasoning effort, allowing applications to select low, medium, or high reasoning depending on the required balance between reasoning depth and inference latency. It also supports agentic capabilities including function calling, web browsing, Python execution, and structured outputs.

---

Model Overview

• Model Name: GPT-OSS-20B
• Base Model: openai/gpt-oss-20b
• Architecture: Mixture-of-Experts Causal Language Model
• Parameter Count: 20.9B total parameters
• Active Parameters: 3.6B per token
• Number of Experts: 32
• Active Experts: 4 per token
• Context Window: 128K tokens
• Modalities: Text
• Primary Language: Multilingual
• Developer: OpenAI
• License: Apache 2.0

---

Quantization Formats

This repository provides GGUF-converted versions of the GPT-OSS-20B model optimized for efficient local inference. The supplied conversion contains an F16 representation together with IQ3_M, IQ4_NL, IQ4_XS, and Q6_K quantization variants.

IQ3_M

• Aggressive 3-bit quantization designed to reduce the storage footprint of the GPT-OSS-20B model
• Provides a lower-memory representation for local deployment and resource-constrained inference environments
• Suitable for experimentation with reasoning, coding, conversational generation, and agentic workloads where memory efficiency is important
• The lower numerical precision can introduce greater quantization-related changes in detailed reasoning and generation behavior compared with higher-precision representations

IQ4_NL

• 4-bit non-linear quantization designed to provide a higher-precision representation than the supplied IQ3_M variant
• Provides a practical balance between model storage requirements and preservation of the underlying model behavior
• Suitable for reasoning, coding, structured generation, and tool-oriented workloads requiring a more precise representation than the 3-bit variant
• Requires slightly more storage than IQ3_M while retaining a 4-bit representation

IQ4_XS

• Compact 4-bit quantization providing an alternative deployment point between aggressive compression and higher-precision representations
• Designed to reduce model storage requirements while retaining useful reasoning and generation behavior
• Suitable for local inference, coding, agentic workflows, and general text-generation workloads
• Provides a slightly smaller footprint than IQ4_NL while maintaining the same nominal 4-bit precision class

---

Training Overview

GPT-OSS-20B is an autoregressive Mixture-of-Experts Transformer developed with an emphasis on reasoning, efficiency, tool use, and practical deployment. The architecture contains 24 layers, 32 total experts, and activates 4 experts for each token.

Pretraining

The model was trained primarily on text data with emphasis on STEM, coding, and general knowledge. The architecture uses alternating sliding-window and full-attention patterns, Grouped Query Attention (GQA), and Rotary Position Embeddings (RoPE).

Training focuses on:

• Large-scale language modeling
• General knowledge acquisition
• Mathematical and STEM reasoning
• Software and coding knowledge
• Long-context language processing
• Tool-oriented reasoning
• Efficient sparse Mixture-of-Experts computation

Post-Training

The model underwent supervised fine-tuning and high-compute reinforcement learning to improve reasoning and real-world task performance. OpenAI describes the post-training process as teaching the model to apply chain-of-thought reasoning and tool use before producing its final answer.

Post-training focuses on:

• Instruction following
• Configurable reasoning behavior
• Agentic task execution
• Function calling
• Web browsing
• Python code execution
• Structured outputs
• Coding and technical workflows
• Multi-step problem solving

The released GPT-OSS models were post-trained with MXFP4 quantization of the Mixture-of-Experts weights. OpenAI reports that GPT-OSS-20B can run within approximately 16 GB of memory using its native MXFP4 representation.

---

Core Capabilities

• Advanced Reasoning Supports multi-step reasoning across mathematical, analytical, technical, and knowledge-intensive tasks.

• Agentic Task Execution Designed for workflows involving planning, tool interaction, external environments, and multi-step task completion.

• Configurable Reasoning Provides low, medium, and high reasoning-effort configurations for balancing response quality and latency.

• Function Calling Supports structured interaction with external functions and tools, enabling integration into agentic applications.

• Coding and Software Engineering Performs code generation, debugging, code analysis, and software-development-oriented reasoning.

• Web and Tool Use Supports agentic workflows involving web browsing and external tools.

• Python Execution Can participate in workflows where Python execution is provided as an external tool.

• Structured Outputs Supports generation patterns suitable for applications requiring machine-readable responses.

• Long-Context Processing Supports context lengths of up to 128K tokens, making it suitable for long documents, codebases, conversations, and extended agentic workflows.

• Fine-Tuning Designed to support customization for specialized applications through parameter fine-tuning.

---

Example Usage

llama.cpp

./llama-cli \
  -m SandLogicTechnologies/GPT-OSS-20B-IQ4_NL.gguf \
  -p "Explain the architectural differences between dense and Mixture-of-Experts language models."

---

Recommended Use Cases

• Advanced reasoning and analytical AI systems
• Agentic AI and autonomous task workflows
• Coding and software-engineering assistants
• Function-calling applications
• Tool-augmented AI systems
• Mathematical and STEM problem solving
• Structured information generation
• Research and knowledge-intensive workflows
• Local AI assistants
• Enterprise workflow automation
• Long-context document analysis
• Customized and fine-tuned domain-specific applications
• Multi-step planning and task execution

---

Acknowledgments

These quantized models are based on the original work by the OpenAI development team.

Special thanks to:

• The OpenAI team for developing and releasing the GPT-OSS-20B model.

• The OpenAI team for releasing the GPT-OSS architecture, Harmony response format, reasoning controls, and open-weight model implementations.

• Georgi Gerganov and the llama.cpp open-source community for enabling efficient quantization and inference via the GGUF format.

---

Contact

For any inquiries or support, please contact us at [email protected] or visit our Website.