gpt-oss-20B-FP8

GPT-OSS 20B — FP8 (block-wise GPTQ)

FP8 (E4M3) post-training quantization of GPT-OSS 20B, produced with block-wise GPTQ (Hessian-based, second-order weight quantization) on a parallel-Hessian calibration pass. Quantized from unsloth/gpt-oss-20b-BF16.

Method

• Algorithm: block-wise GPTQ, blocksize 128, percdamp 0.01
• Calibration: C4, 512 sequences × 2048 tokens (parallel Hessian collection)
• Format: FP8 E4M3, per-channel. The quantization error is baked into the weights, which are stored in bfloat16 (simulated FP8), so the file size is ~40 GB and the model loads with a standard from_pretrained call.

Results (FP8 vs BF16, seq_len 2048)

FP8 matches the BF16 baseline within threshold on both datasets.

The absolute perplexity is inflated by a long-context degradation in GPT-OSS's 2048-token windowed evaluation. This affects the BF16 and models identically, so Δppl is the meaningful metric — the absolute values are not comparable across models.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("anirudmohan26/gpt-oss-20
tok   = AutoTokenizer.from_pretrained("anirudmohan26/gpt-oss-20B-FP8")