Downloads · 30 days
82
10% of all-time downloads
sandeshrajx/gpt-oss-20b-reap-0.4-mxfp4-gguf
gpt-oss-20b-reap-0.4-mxfp4-gguf is a text generation model from sandeshrajx. Use it when you need the model to write or continue text.
This repository contains a GGUF quantized version of the sandeshrajx/gpt-oss-20b-reap-0.4-mxfp4 model.
Downloads · 30 days
82
10% of all-time downloads
All-time downloads
815
Public
Repo size
8.3 GB
Likes
2
Public
Click a slice to open those files.
.gguf8.3 GB · 100%
From the Hugging Face model README
This repository contains a GGUF quantized version of the sandeshrajx/gpt-oss-20b-reap-0.4-mxfp4 model.
This model is a GGUF quantized version of the MXFP4 quantized openai/gpt-oss-20b model.
openai/gpt-oss-20breap with a compression ratio of 0.4llama.cpptheblackcat102/evol-codealpaca-v1The original MXFP4 quantization specifically targeted the "expert" layers of the model, skipping self-attention and router layers, as is standard practice for Mixture-of-Experts (MoE) models to optimize performance and reduce size. This GGUF quantization further reduces the model size for efficient inference with llama.cpp.
You can use this model with llama.cpp or compatible GGUF loaders.
The model was first pruned with a 0.4 compression ratio using reap, then quantized to MXFP4. Subsequently, it was converted to GGUF (Q8_0) format using the llama.cpp conversion script.
Pruning Commands Used:
python ./reap/src/reap/prune.py \
--model-name "openai/gpt-oss-20b" \
--run_observer_only true \
--samples_per_category 32
python ./reap/src/reap/prune.py \
--model-name "openai/gpt-oss-20b" \
--compression-ratio 0.4 \
--prune-method reap
MXFP4 Quantization Command Used:
python Model-Optimizer/examples/gpt-oss/convert_oai_mxfp4_weight_only.py \
--model_path /workspace/artifacts/gpt-oss-20b/evol-codealpaca-v1/pruned_models/reap-seed_42-0.4-mxfp4 \
--output_path /workspace/artifacts/gpt-oss-20b/evol-codealpaca-v1/pruned_models/reap-seed_42-0.4-mxfp4-quantized
GGUF Quantization Command Used:
python llama.cpp/convert_hf_to_gguf.py \
--outtype q8_0 \
--outfile /path/to/output/gpt-oss-20b-reap-0.4-mxfp4-q8_0.gguf \
/path/to/downloaded/mxfp4_model
(Please specify the license of the original model and any modifications)