Downloads · 30 days
0
marcusmi4n/abeja-qwen2.5-7b-japanese-qnn
abeja-qwen2.5-7b-japanese-qnn is a text generation model from marcusmi4n. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This repository contains the ABEJA Qwen 2.5 7B Japanese model optimized for Qualcomm Neural Network (QNN) deployment.
Downloads · 30 days
0
Access
Public
Updated Sep 2, 2025
Repo size
20.9 GB
Likes
1
Public
Click a slice to open those files.
.safetensors20.8 GB · 99%
From the Hugging Face model README
This repository contains the ABEJA Qwen 2.5 7B Japanese model optimized for Qualcomm Neural Network (QNN) deployment.
quantized_simple/onnx/prefill/model.onnx - Context prefilltoken_gen/model.onnx - Token generationquantized_onnx/qnn_compiled/from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("marcusmi4n/abeja-qwen2.5-7b-japanese-qnn", subfolder="quantized_simple")
tokenizer = AutoTokenizer.from_pretrained("marcusmi4n/abeja-qwen2.5-7b-japanese-qnn", subfolder="quantized_simple")
# Japanese text generation
inputs = tokenizer("こんにちは、私は", return_tensors="pt")
outputs = model.generate(**inputs, max_length=100, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
import onnxruntime as ort
# Load ONNX model
session = ort.InferenceSession("marcusmi4n/abeja-qwen2.5-7b-japanese-qnn/onnx/prefill/model.onnx")
# Run inference...
# Deploy to Snapdragon device
adb push marcusmi4n/abeja-qwen2.5-7b-japanese-qnn/qnn_compiled/ /data/local/tmp/qnn_model/
# Use QNN runtime for inference
marcusmi4n/abeja-qwen2.5-7b-japanese-qnn/
├── quantized_simple/ # 4-bit quantized PyTorch model
│ ├── model.safetensors
│ ├── config.json
│ ├── tokenizer.json
│ └── model_info.json
├── onnx/ # ONNX models
│ ├── prefill/model.onnx
│ └── token_gen/model.onnx
├── quantized_onnx/ # Quantized ONNX models
│ ├── prefill/model_quantized.onnx
│ └── token_gen/model_quantized.onnx
├── qnn_compiled/ # QNN compiled models
│ ├── prefill/
│ ├── token_gen/
│ └── deployment_info.json
└── README.md # This file
Apache 2.0 - Same as base ABEJA Qwen 2.5 model
@misc{abeja-qwen25-qnn,
title={ABEJA Qwen 2.5 7B Japanese - QNN Optimized},
author={QNN Conversion Pipeline},
year={2025},
url={https://huggingface.co/marcusmi4n/abeja-qwen2.5-7b-japanese-qnn}
}
Please cite the original ABEJA Qwen 2.5 paper:
@article{abeja-qwen2.5,
title={ABEJA Qwen 2.5: Japanese Language Model},
author={ABEJA Inc.},
journal={arXiv preprint},
year={2024}
}