Downloads · 30 days
9
1% of all-time downloads
kaptaan45/QaptaanLM-0.75B-ONNX
QaptaanLM-0.75B-ONNX is a text generation model from kaptaan45. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
[](https://opensource.org/licenses/Apache-2.0) [](https://huggingface.co/kaptaan45/QaptaanLM-0.75B) [](https://github.com/rudy-07/QaptaanLM-0.75B)
Downloads · 30 days
9
1% of all-time downloads
All-time downloads
1K
Public
Repo size
4 GB
Likes
0
Public
Click a slice to open those files.
Other3 GB · 74%
From the Hugging Face model README
This repository contains exported ONNX Runtime graph weights for QaptaanLM-0.75B (Base CPT foundation model), designed for lightweight in-browser IDE autocompletion and edge device code inference.
onnxruntimeimport os
import numpy as np
import onnxruntime as ort
from huggingface_hub import snapshot_download
from transformers import AutoTokenizer
model_id = "kaptaan45/QaptaanLM-0.75B-ONNX"
model_dir = snapshot_download(repo_id=model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
session = ort.InferenceSession(
os.path.join(model_dir, "model.onnx"),
providers=["CUDAExecutionProvider", "CPUExecutionProvider"]
)
prompt = 'def is_palindrome(s: str) -> bool:\n \"\"\"Return True if s is a palindrome.\"\"\"\n '
input_ids = tokenizer.encode(prompt, return_tensors="np")
outputs = session.run(None, {"input_ids": input_ids})
logits = outputs[0]
next_token = int(np.argmax(logits[0, -1, :]))
print("Next Token ID:", next_token, "Decoded:", repr(tokenizer.decode([next_token])))
Released under the Apache 2.0 License.