gpt-oss-puzzle-88B-GGUF

gpt-oss-puzzle-88B-GGUF

EXPERIMENTAL - REQUIRES CUSTOM BRANCH

These GGUF files will NOT work with mainline llama.cpp. You must use the branch linked below.

GGUF quantisation of nvidia/gpt-oss-puzzle-88B, an 88B parameter MoE model derived from gpt-oss-120B using NVIDIA's Puzzle NAS framework.

Required Branch

This model requires a custom llama.cpp branch with gpt-oss-puzzle architecture support:

https://github.com/smpurkis/llama.cpp/tree/gpt-oss-puzzle-support

Tracking issue: ggml-org/llama.cpp#21028 PR: ggml-org/llama.cpp#21032

This will not work on mainline llama.cpp until the architecture is merged upstream.

How to Use

# Clone the required branch
git clone --branch gpt-oss-puzzle-support https://github.com/smpurkis/llama.cpp.git
cd llama.cpp

# Build (example with Vulkan)
cmake -B build -DGGML_VULKAN=1
cmake --build build --config Release -j$(nproc)

# Run
./build/bin/llama-cli -m gpt-oss-puzzle-88B.MXFP4_MOE.gguf -ngl 99 -fa 1 -p "Hello"

Available Quantisations

Architecture Differences from gpt-oss-120B

The puzzle model differs from the standard gpt-oss architecture in ways that require dedicated support:

Credits

• Original model: NVIDIA
• llama.cpp architecture support: smpurkis/llama.cpp@gpt-oss-puzzle-support