Downloads · 30 days
0
marcusmi4n/abeja-qwen2.5-7b-japanese-onnx
abeja-qwen2.5-7b-japanese-onnx is a text generation model from marcusmi4n. Use it when you need the model to write or continue text. It is set up for onnx. The card lists the license as apache-2.0.
This repository contains ONNX models for the ABEJA Qwen 2.5 7B Japanese model, optimized for cross-platform inference. The models are split into prefill and token generation components for optimal performance.
Downloads · 30 days
0
Access
Public
Updated Sep 1, 2025
Repo size
57.3 MB
Likes
0
Public
Click a slice to open those files.
.onnx57.3 MB · 100%
From the Hugging Face model README
This repository contains ONNX models for the ABEJA Qwen 2.5 7B Japanese model, optimized for cross-platform inference. The models are split into prefill and token generation components for optimal performance.
prefill/model.onnxtoken_gen/model.onnximport onnxruntime as ort
import numpy as np
# Load models
prefill_session = ort.InferenceSession('prefill/model.onnx')
token_gen_session = ort.InferenceSession('token_gen/model.onnx')
# Example inference
input_ids = np.array([[1, 2, 3, 4, 5]], dtype=np.int64)
prefill_outputs = prefill_session.run(None, {"input_ids": input_ids})
token_outputs = token_gen_session.run(None, {"hidden_states": prefill_outputs[0]})
#include <onnxruntime_cxx_api.h>
Ort::Env env(ORT_LOGGING_LEVEL_WARNING, "test");
Ort::Session session(env, "prefill/model.onnx", Ort::SessionOptions{{nullptr}});
// Run inference
std::vector<int64_t> input_shape = {1, 5};
std::vector<int64_t> input_data = {1, 2, 3, 4, 5};
auto memory_info = Ort::MemoryInfo::CreateCpu(OrtArenaAllocator, OrtMemTypeDefault);
Ort::Value input_tensor = Ort::Value::CreateTensor<int64_t>(
memory_info, input_data.data(), input_data.size(), input_shape.data(), input_shape.size());
# CPU version
pip install onnxruntime
# GPU version (NVIDIA)
pip install onnxruntime-gpu
# For mobile deployment
pip install onnxruntime-mobile
このリポジトリには、クロスプラットフォーム推論用に最適化されたABEJA Qwen 2.5 7B日本語モデルのONNXモデルが含まれています。モデルは最適なパフォーマンスのためにプレフィルとトークン生成コンポーネントに分割されています。
prefill/model.onnxtoken_gen/model.onnximport onnxruntime as ort
import numpy as np
# モデルを読み込み
prefill_session = ort.InferenceSession('prefill/model.onnx')
token_gen_session = ort.InferenceSession('token_gen/model.onnx')
# 推論例
input_ids = np.array([[1, 2, 3, 4, 5]], dtype=np.int64)
prefill_outputs = prefill_session.run(None, {"input_ids": input_ids})
token_outputs = token_gen_session.run(None, {"hidden_states": prefill_outputs[0]})
# CPU版
pip install onnxruntime
# GPU版(NVIDIA)
pip install onnxruntime-gpu
# モバイルデプロイ用
pip install onnxruntime-mobile
Author: Mukwaya Mark