Qwen3-Coder-30B-A3B-Instruct-FP8-EAGLE3

Qwen3-Coder-30B-A3B-Instruct-FP8-EAGLE3

Model Introduction

The Eagle3 draft model was trained using the SpecForge framework for the Qwen3-Coder-30B-A3B-Instruct-FP8 model, leveraging a combination of OPC datasets.

Benchmark Results

• humaneval (164 questions)

{
    "humaneval": [
        {
            "batch_size": 1,
            "steps": null,
            "topk": null,
            "num_draft_tokens": null,
            "metrics": [
                {
                    "latency": 409.72047470510006,
                    "output_throughput": 222.746983942377,
                    "accept_length": 5.334580313303718,
                    "accuracy": 0.0,
                    "num_questions": 164,
                    "num_valid_predictions": 164,
                    "categorical_performance": null
                }
            ],
            "num_samples": null
        }
    ]
}

• mtbench (80 questions)

{
    "mtbench": [
        {
            "batch_size": 1,
            "steps": null,
            "topk": null,
            "num_draft_tokens": null,
            "metrics": [
                {
                    "latency": 473.0768761308864,
                    "output_throughput": 160.5087964159795,
                    "accept_length": 3.4988941111418304,
                    "accuracy": null,
                    "num_questions": 80,
                    "num_valid_predictions": 0,
                    "categorical_performance": null
                }
            ],
            "num_samples": null
        }
    ]
}

Usage

You can use this Eagle3 draft model in SGLang with the following command:

python3 -m sglang.launch_server \
    --model <Qwen3-Coder-30B-A3B-Instruct-FP8> \
    --speculative-algorithm EAGLE3 \
    --speculative-draft-model-path <Qwen3-Coder-30B-A3B-Instruct-FP8-EAGLE3> \
    --speculative-num-steps 5 \
    --speculative-eagle-topk 8 \
    --speculative-num-draft-tokens 32 \
    --mem-fraction-static 0.75 \
    --tp 8 \
    --enable-ep-moe \
    --context-length 128000 \
    --trust-remote-code \
    --host 0.0.0.0 \
    --port 30000 \
    --dtype bfloat16