Downloads · 30 days
4
2% of all-time downloads
NghiaNguyen1529/octen-embedding-0.6b-directml-onnx
octen-embedding-0.6b-directml-onnx is a feature extraction model from NghiaNguyen1529. Use it when you need embeddings to search or compare text. It is set up for sentence-transformers. The card lists the license as apache-2.0.
Patched ONNX export of Octen/Octen-Embedding-0.6B that runs on DirectML (Windows GPU via ONNX Runtime).
Downloads · 30 days
4
2% of all-time downloads
All-time downloads
237
Public
Repo size
1.2 GB
Likes
0
Public
Click a slice to open those files.
.data1.2 GB · 99%
From the Hugging Face model README
Patched ONNX export of Octen/Octen-Embedding-0.6B that runs on DirectML (Windows GPU via ONNX Runtime).
The original torch.onnx.export (dynamo) produces val_41 = [-1] used in Reshape shapes for multi-head attention (GQA: 16 Q heads, 8 KV heads). DirectML's execution provider cannot resolve symbolic -1 at graph-capture time.
Fix: Replace [-1] with four concrete head-count constants (16 for Q, 8 for K, 8 for V, 2048 for attention output) and reconnect 84 Reshape consumer nodes.
See fix_octen_dml.py for the full patch script.
model.fp16.onnx — ONNX graph proto (4 MB)model.fp16.onnx.data — external weights (1.1 GB, fp16)tokenizer.json — Qwen2 tokenizerconfig.json — model config (max_position_embeddings=32768)fix_octen_dml.py — reproduction scriptimport onnxruntime as ort
session = ort.InferenceSession(
"model.fp16.onnx",
providers=["DmlExecutionProvider"],
)
| Dataset | R@5 | R@10 | MRR |
|---|---|---|---|
| esp32 (smoke) | 0.930 | 0.950 | 0.810 |
| autosar | 0.678 | 0.774 | 0.552 |
Identical to CPU fp16 reference — patch preserves quality exactly.