qwen3-8b-eagle3-rust

Rust EAGLE-3 — scale400

Strongest Rust candidate tested: 15.63% lower paired fresh-Rust latency vs upstream (14.26–17.11% exploratory interval), and 1.76% vs the matched 512/400 control (0.40–2.84%).

This is a BF16 EAGLE-3 draft for frozen Qwen3-8B, not a standalone language model. It is not quantized and does not improve the target's coding capability. One transformer block, width 4,096; 399,523,840 stored parameters excluding vocabulary buffers. The upstream draft vocabulary mapping is retained.

Training

Pool: 2,048 Rust. Optimizer updates: 400. Seed: 20260918. Batch 1 × accumulation 8; peak LR 1e-5; cosine horizon 400; gradient clip 0.5; training length 7. Checkpoint selected by separate Rust validation simulated acceptance length, not test timing. Target weights remain frozen.

Target: Qwen/Qwen3-8B at b968826d9c46dd6066d109eabc6255188de91218. Warm start: Tengyunw/qwen3_8b_eagle3 at 2a1059d51f622b8cad7d7d72840153ffea5488a0. SpecForge: ed64d275bac8a48e126adb2827368603566a3029; observed SGLang 0.5.18.

Rust prompts: Fortytwo-Network/Strandset-Rust-v1 at 0a8d223302712a2b34a6ad4ce1fd679031894b3d (Apache-2.0). Answers regenerated by Qwen; crate-disjoint splits. This Rust-only checkpoint does not use Dolly. See included source cards and NOTICE for attribution.

Intended use and limits

Research serving experiments on H100 80GB with the pinned target and compatible SGLang. Tested: BF16, three speculative steps, top-k one, four draft slots, temperature zero, thinking off, prefix cache off, 1,024 output-token cap, concurrency one. See reproduction instructions for the full environment and benchmark workflow.

Results are conditional on small synthetic prompt cohorts, not all Rust programs. Rust-only 400-update candidates slowed a small JSON diagnostic. The mixed candidate did not establish universal non-regression. Pilot batch-concurrency output differences remain unresolved. No functional code benchmark or stochastic-distribution test was performed. Do not treat an acceptance proxy as measured latency or a frozen target as sufficient proof of output parity.

Download and serve

Full experiment and code · Readable write-up · Other main draft

Use a compatible GPU environment with the observed SGLang 0.5.18 stack. The measured hardware was one H100 80GB. This repository is a draft component, not a standalone Transformers text-generation pipeline. The original cached environment was tested; a clean GPU rebuild was not revalidated for this upload.

Download both the frozen target and this draft, verify the original weight checksums, and start the server:

export QWEN_TARGET_PATH="$(hf download Qwen/Qwen3-8B --revision b968826d9c46dd6066d109eabc6255188de91218)"
# This immutable revision contains the released weights.
export RUST_DRAFT_PATH="$(hf download RSRS64/qwen3-8b-eagle3-rust --revision 888e7a454f370ec69bdc2abadc5ea80c57233589)"
python - <<'CHECK'
import hashlib, json, os
from pathlib import Path
p = Path(os.environ["RUST_DRAFT_PATH"])
for name, expected in json.loads((p / "SHA256.json").read_text()).items():
    with (p / name).open("rb") as f:
        assert hashlib.file_digest(f, "sha256").hexdigest() == expected, name
print("Draft checksums verified")
CHECK
python -m sglang.launch_server \
  --model-path "$QWEN_TARGET_PATH" --served-model-name rust-target \
  --host 127.0.0.1 --port 30000 --dtype bfloat16 \
  --context-length 8192 --mem-fraction-static 0.7 --random-seed 20260918 \
  --max-running-requests 16 --disable-radix-cache --enable-metrics \
  --speculative-algorithm EAGLE3 \
  --speculative-draft-model-path "$RUST_DRAFT_PATH" \
  --speculative-num-steps 3 --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4

For the measured setting, send requests at concurrency one, temperature zero, with chat_template_kwargs={"enable_thinking": false} and a 1,024-token output cap. The complete benchmark client and fixed prompts are in the experiment repository. Serving requires the target weights and tokenizer downloaded above; no target model files are included here.

Files and terms

model.safetensors: 799,457,128 bytes. SHA-256: 8084d09ddbd72c6874c81793534090e5a58ad69c7eefb22a8c5e7ddca4fd6b36. config.json and SHA256.json accompany the weights. The checkpoint does not include target weights, tokenizer, optimizer state, or feature tensors.

Derived from the upstream MIT-declared draft; its exact model card and the target's Apache license are included. Original adaptation contributions use the MIT license. Dataset source material retains its own terms; NOTICE.md and licenses/ preserve attribution. No blanket relicensing of third-party material is intended.