gpt-oss-20b-speculator.eagle3

gpt-oss-20b-speculator.eagle3

Model Overview

• Verifier: openai/gpt-oss-20b
• Speculative Decoding Algorithm: EAGLE-3
• Model Architecture: Eagle3Speculator
• Release Date: 04/02/2026
• Version: 3.0
• Model Developers: RedHat

This is a speculator model designed for use with openai/gpt-oss-20b, based on the EAGLE-3 speculative decoding algorithm. It was trained using the speculators library on hidden states distilled from openai/gpt-oss-120b, using the inference-optimization/gpt-oss-120b dataset (a combination of Magpie-Align/Magpie-Llama-3.1-Pro-300K-Filtered and the train_sft split of HuggingFaceH4/ultrachat_200k). This model should be used with the openai/gpt-oss-20b chat template, specifically through the /chat/completions endpoint.

Use with vLLM

vllm serve openai/gpt-oss-20b \
  -tp 1 \
  --speculative-config '{
    "model": "RedHatAI/gpt-oss-20b-speculator.eagle3",
    "num_speculative_tokens": 3,
    "method": "eagle3"
  }'

Evaluations

Use cases

Acceptance lengths

<details> <summary>Details</summary> <strong>Configuration</strong>

• temperature: default (greedy)
• repetitions: 1
• time per experiment: 5min
• hardware: 1xA100
• vLLM version: 0.17.1
• GuideLLM version: 0.5.5

<strong>Command</strong>

GUIDELLM__MAX_CONCURRENCY=128 \
GUIDELLM__PREFERRED_ROUTE="chat_completions" \
guidellm benchmark \
  --target "http://localhost:8000/v1" \
  --data "RedHatAI/speculator_benchmarks" \
  --data-args '{"data_files": "HumanEval.jsonl"}' \
  --rate-type throughput \
  --max-seconds 300 \
  --output-path "gpt-oss-20b-HumanEval.json"

GuideLLM interface changed, so for compatibility with the latest version (v0.6.0), please use the following command:

GUIDELLM__PREFERRED_ROUTE="chat_completions" \
guidellm benchmark \
  --target "http://localhost:8000/v1" \
  --data "RedHatAI/speculator_benchmarks" \
  --data-args '{"data_files": "HumanEval.jsonl"}' \
  --profile sweep \
  --max-seconds 1800 \
  --output-path "my_output.json" \
  --backend-args '{"extras": {"body": {"temperature":0.6, "top_p":0.95, "top_k":20}}}'

</details>