onnx-genai-example-qwen3-0-6b-eagle3

onnx-genai-example-qwen3-0-6b-eagle3

Public real-weight chained speculative package combining Qwen/Qwen3-0.6B and Nicolassuez/Qwen3-0.6B-eagle3. Both sources are Apache-2.0.

The package contains real fp16 target/proposer weights, target embeddings, exact draft-to-target vocabulary mapping, tokenizer, canonical model-agnostic chained-proposer metadata, provenance, and real CUDA evidence. The ONNX GenAI L4/L5 probe generated exactly the same 12 target tokens as greedy baseline while accepting 6 and rejecting 27 proposals.

Download

hf download justinchuby/onnx-genai-example-qwen3-0-6b-eagle3 --repo-type model --local-dir ./qwen3-0.6b-eagle3

Exact runtime test

export ONNX_GENAI_CHAINED_SPEC_PACKAGE="$PWD/qwen3-0.6b-eagle3"
export ONNX_GENAI_ORT_LIB=/path/to/libonnxruntime.so
export LD_LIBRARY_PATH="$(dirname "$ONNX_GENAI_ORT_LIB"):$LD_LIBRARY_PATH"
export ONNX_GENAI_EP=cuda ONNX_GENAI_EP_FALLBACK=1
cargo test -p onnx-genai-engine --features ort-cuda --test chained_proposer_real -- --nocapture

runtime-target/ is the executable target view used by the test. inference_metadata.yaml is the canonical workflow contract; runtime-target/inference_metadata.mobius.yaml preserves the original Mobius export metadata.

<!-- inference-metadata-annotation:start -->

Annotated inference metadata

Review inference_metadata.annotated.yaml for inline explanations of this package's workflow, tensor/state/cache contracts, and fail-closed omissions. inference_metadata.yaml remains the canonical machine-authored contract; automated validation confirms both files parse to the same metadata object.

<!-- inference-metadata-annotation:end -->