Downloads · 30 days
17
14% of all-time downloads
Ringoacid/Qwen2.5-0.5B-onnx-attention-int8
Qwen2.5-0.5B-onnx-attention-int8 is a text generation model from Ringoacid. Use it when you need the model to write or continue text. It is set up for onnxruntime. The card lists the license as apache-2.0.
This is an ONNX-converted, 8-bit dynamically quantized version of Qwen/Qwen2.5-0.5B, packaged for in-browser inference via ONNX Runtime Web with the WebGPU execution provider.
Downloads · 30 days
17
14% of all-time downloads
All-time downloads
122
Public
Repo size
513 MB
Likes
0
Public
Click a slice to open those files.
.data988 MB · 98%
From the Hugging Face model README
This is an ONNX-converted, 8-bit dynamically quantized version of
Qwen/Qwen2.5-0.5B, packaged
for in-browser inference via ONNX Runtime Web
with the WebGPU execution provider.
Original model: Qwen/Qwen2.5-0.5B (Apache-2.0)
| File | Purpose | Size |
|---|---|---|
analyze_int8.onnx (+ .data) | Single forward pass with per-layer attention weights as graph outputs (eager attention, no KV cache) — used by the visualizer's "analyze" mode | ~476 MB |
generate_int8.onnx (+ .data) | Merged prefill+decode model with KV cache (SDPA, no attention output) — used by the visualizer's "generate" mode | ~474 MB |
tokenizer.json, vocab.json, merges.txt, etc. | Qwen2 BPE tokenizer (Transformers.js compatible) | ~16 MB |
analyze_int8.onnxinput_ids[B,S], attention_mask[B,S] (both int64)logits[B,S,151936], attentions.0..23[B,14,S,S]generate_int8.onnxinput_ids[B,S], attention_mask[B,P+S], position_ids[B,S],
and 24x past_key_values.{i}.{key,value}[B,2,P,64]logits[B,S,151936], 24x present.{i}.{key,value}[B,2,P+S,64]Where B=batch (1), S=current sequence length, P=past sequence length.
QInt8, per-channelonnxruntime.quantization.quantize_dynamicimport * as ort from 'onnxruntime-web';
const base = 'https://huggingface.co/Ringoacid/Qwen2.5-0.5B-onnx-attention-int8/resolve/main';
const sess = await ort.InferenceSession.create(`${base}/analyze_int8.onnx`, {
executionProviders: ['webgpu', 'wasm'],
graphOptimizationLevel: 'all',
});
Apache-2.0, inherited from the base model.