Downloads · 30 days
0
anthonypjshaw/harrier-oss-v1-270m-qnn-htp
harrier-oss-v1-270m-qnn-htp is a feature extraction model from anthonypjshaw. Use it when you need embeddings to search or compare text. It is set up for onnxruntime. The card lists the license as mit.
This is a static-INT8 (int8) quantized build of microsoft/harrier-oss-v1-270m, compiled by Qualcomm AI Hub to a precompiled QNN context binary wrapped in an ONNX file consumable by ONNX Runtime's QNN Execution Provide…
Downloads · 30 days
0
Access
Public
Updated May 18, 2026
Repo size
475 MB
Likes
0
Public
Click a slice to open those files.
.bin441 MB · 93%
From the Hugging Face model README
This is a static-INT8 (int8) quantized build of
microsoft/harrier-oss-v1-270m,
compiled by Qualcomm AI Hub to a precompiled QNN
context binary wrapped in an ONNX file consumable by ONNX Runtime's QNN
Execution Provider on Snapdragon X Elite (Hexagon HTP V81).
The 270M variant is the smaller, lower-latency sibling of
harrier-oss-v1-0.6b
built on the Gemma3-text backbone. Intended for local, on-device embedding
generation.
microsoft/harrier-oss-v1-270m
(~540 MB safetensors, Gemma3-text architecture).optimum.exporters.onnx.main_export( library_name="sentence_transformers", task="feature-extraction", no_dynamic_axes=True, batch_size=1, sequence_length=512, opset=17).
This bakes the full Sentence-Transformers pipeline (backbone + masked
mean pool + L2 normalise) into a single ONNX graph that uses only
ai.onnx ops and contains no GatherND nodes.SymbolicShapeInference + onnxsim.simplify to
fold constants and propagate shape info before upload (AI Hub's converter
env doesn't ship with onnxruntime or onnxsim).submit_compile_job call with:
- --target_runtime precompiled_qnn_onnx
- --quantize_full_type int8 (static INT8 weights + activations)
- --truncate_64bit_io (HTP I/O is bounded to int32)
- calibration_data = 20 tokenized representative natural-language inputs
targeting Snapdragon X Elite CRD running OS 11.Because the PT -> ONNX export is done locally with optimum's
sentence-transformers OnnxConfig (not AI Hub's dynamo converter), the
resulting graph reaches the Hexagon NPU end-to-end without the GatherND
op that previously broke QNN's CPU validator.
| File | Purpose |
|---|---|
harrier-oss-v1-270m.qnn_ctx.onnx | ONNX wrapper containing the embedded QNN HTP context binary |
tokenizer.json | HF Tokenizers JSON (copied verbatim from the source repo) |
tokenizer_config.json, special_tokens_map.json | Tokenizer metadata |
aihub-metadata.txt | URLs of the AI Hub compile / profile jobs |
import * as ort from "onnxruntime-node";
const session = await ort.InferenceSession.create(
"harrier-oss-v1-270m.qnn_ctx.onnx",
{
executionProviders: [
{
name: "qnn",
backend_path: "QnnHtp.dll",
htp_performance_mode: "burst",
htp_graph_finalization_optimization_mode: "3",
},
],
graphOptimizationLevel: "all",
},
);
const inputIds = new BigInt64Array(512).fill(0n);
const attentionMask = new BigInt64Array(512).fill(1n);
const feeds = {
input_ids: new ort.Tensor("int64", inputIds, [1, 512]),
attention_mask: new ort.Tensor("int64", attentionMask, [1, 512]),
};
const { embedding } = await session.run(feeds);
| Concern | 270M (this) | 0.6B |
|---|---|---|
| Latency on HTP | Lower (smaller graph) | Higher |
| Embedding quality | Lower (smaller model) | Higher |
| Peak memory | Lower | Higher |
| Best for | Always-on background tasks, embedded scenarios | Foreground retrieval where quality matters |
512 tokens. Pad or truncate accordingly.MIT, inherited from
microsoft/harrier-oss-v1-270m.