Qwen2.5-0.5B-Instruct-ONNX

Qwen2.5-0.5B-Instruct — ONNX

ONNX export of Qwen/Qwen2.5-0.5B-Instruct, published by Liodon AI. Exported with optimum (optimum.exporters.onnx.main_export, task text-generation-with-past, so the graph exposes past-key-value inputs/outputs for KV-cached autoregressive decoding).

Files

Quick Start

import onnxruntime as ort
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("liodon-ai/Qwen2.5-0.5B-Instruct-ONNX")
sess = ort.InferenceSession("model_quantized.onnx", providers=["CPUExecutionProvider"])
# past_key_values.*.key / .value inputs must be supplied (zero-length tensors
# for the first forward pass) -- see optimum's ORTModelForCausalLM for a
# ready-made wrapper that handles KV-cache bookkeeping automatically:
#   from optimum.onnxruntime import ORTModelForCausalLM
#   model = ORTModelForCausalLM.from_pretrained("liodon-ai/Qwen2.5-0.5B-Instruct-ONNX", file_name="model_quantized.onnx")

Source

• Model: Qwen/Qwen2.5-0.5B-Instruct
• License: apache-2.0

---

Exported by Liodon AI