Tess-4-27B-EAGLE3

Tess-4-27B · EAGLE-3 Draft

An EAGLE-3 speculative-decoding draft head for migtissera/Tess-4-27B — trained on-policy against Tess-4's own outputs, so it predicts this model, not its base.

Measured: 1.76× faster decoding on average — up to 2.4× on reasoning. Lossless: the target model verifies every token, so outputs are identical to non-speculative decoding.

Benchmarks

Single H100-80GB, greedy decoding, 10 held-out prompts × 1024 tokens each:

Server-reported accept length: 1.5–2.2 tokens per verification step (conservative settings: topk=1, 4 draft tokens — tuning headroom remains).

Why this exists

Tess-4-27B inherits an MTP head from its base model, but fine-tuning shifts the output distribution, driving that head's acceptance to ~0% — it can't accelerate the fine-tune. This EAGLE-3 draft was trained against Tess-4 itself: ~4,700 on-policy samples (Tess-4's own reasoning + coding + agentic generations, including real multi-turn tool-call traces), 2 epochs, with the draft consuming fused hidden states from three depths of the frozen target.

Usage

SGLang (tested ✅)

python -m sglang.launch_server \
  --model-path migtissera/Tess-4-27B \
  --speculative-algorithm EAGLE3 \
  --speculative-draft-model-path migtissera/Tess-4-27B-EAGLE3 \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4 \
  --trust-remote-code

vLLM

vllm serve migtissera/Tess-4-27B \
  --speculative-config '{"method": "eagle3", "model": "migtissera/Tess-4-27B-EAGLE3", "num_speculative_tokens": 4}' \
  --trust-remote-code

(Benchmarked with SGLang; vLLM flags may vary by version.)

Details

• Architecture: 1-layer Llama-style EAGLE-3 head (hidden 5120, matching the target), draft vocab 32K mapped onto the target's 248K vocab. 1.22 GB. Input embeddings are not included — the serving engine shares them from the target model.
• Training: SpecForge (SGLang team), online mode, frozen target, single H100.
• Data: on-policy — the target generated every training token. Mix: agentic/coding prompts from the Tess-4 program, OpenCoder, GSM8K, UltraChat, plus real multi-turn tool-call traces.
• Lossless: speculative decoding never changes outputs; the target verifies every proposed token. Speedup depends on workload (structured/reasoning content accepts more).

License

Apache 2.0, matching Tess-4-27B.

Part of the Tess series by Migel Tissera.