Spark-X2.5-4B-onnx

Spark-X2.5-4B-onnx

ONNX export of XHToken/Spark-X2.5-4B for onnxruntime-web / WebGPU.

Two graphs per precision under onnx/:

• fp16/prefill/model.onnx — prompt tokens -> logits + present KV
• fp16/decode/model.onnx — token + past KV -> logits + updated KV

Weights are ONNX external-data shards (onnx__MatMul_*) sitting next to each model.onnx (the graphs exceed the 2 GB protobuf limit). Keep each graph's directory together when serving.

Architecture

spark2_5 hybrid: 3 sliding-attention layers (window 512, RoPE theta 10k) per 1 full-attention layer (partial rotary 0.25, RoPE theta 5M). Headwise sigmoid attention output gate. GQA 16/4, head_dim 256, tied embeddings, vocab 131072.

Inputs

• input_ids [B, T] int64
• position_ids [B, T] int64 (absolute positions; prefill: 0..T-1; decode: past_len)
• attn_mask [B, T, total] additive float (0 keep / -inf mask). Host builds causal mask; sliding layers additionally mask keys outside the 512-token window.
• past_k_35 / past_v_35 [B, 4, past_len, 256] — zero-length for prefill.

Outputs

• logits [B, T, 131072]
• present_k_35 / present_v_35 [B, 4, total_len, 256]

Loop: prefill once, then feed present_* back as past_* each decode step, slicing position_ids and attn_mask accordingly. Greedy sampling host-side from logits.

WebGPU: onnxruntime-web webgpu EP. fp16 graphs ~9 GB each; int4 weight-only graphs ~2.9 GB each — use int4 for consumer GPUs.