Downloads · 30 days
15
4% of all-time downloads
shalyhinpavel/harrier-v1-270m-gguf
harrier-v1-270m-gguf is a feature extraction model from shalyhinpavel. Use it when you need embeddings to search or compare text. The card lists the license as mit.
This is a quantized GGUF port (Q80) of the Microsoft Harrier-oss-v1-270m multilingual text embedding model. It has been specifically ported and tested for integration with the Sovereign Engine, optimized for extremely…
Downloads · 30 days
15
4% of all-time downloads
All-time downloads
347
Public
Repo size
557 MB
Likes
0
Public
Click a slice to open those files.
.gguf557 MB · 100%
From the Hugging Face model README
This is a quantized GGUF port (Q8_0) of the Microsoft Harrier-oss-v1-270m multilingual text embedding model. It has been specifically ported and tested for integration with the Sovereign Engine, optimized for extremely high-throughput batch inference on CPU and Apple Silicon via the Rust candle framework.
| Parameter | Value |
|---|---|
| Architecture | Decoder-only Autoregressive Transformer |
| Parameters | 270 Million |
| Embedding Dimension | 640 ($f32$, $L_2$ Normalized) |
| Max Context Length | 32,768 tokens |
| Quantization | Q8_0 (8-bit) |
Unlike standard BERT-based sentence transformers, Harrier-oss-v1 utilizes a causal decoder-only architecture. If you are implementing custom inference engines, you must follow these strict architectural rules, otherwise, your retrieval accuracy will drop to near zero.
This model does not use [CLS] token pooling or Mean Pooling. You must extract the embedding from the very last non-padding token (EOS) of the sequence.
The model is instruction-trained. You must apply strict asymmetric prompting for queries vs. documents:
Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: <Your Query>
Standard implementations of Gemma-based architectures often multiply the input embeddings by a scaling factor of $\sqrt{\text{hidden_size}}$ (e.g., $\sqrt{640}$). Do not apply this scaling factor to this specific GGUF export. The weights have been aligned such that applying the $\sqrt{d}$ scalar will completely blow out the attention activations and destroy the latent space representation.
Because it is an autoregressive model, your inference engine will accumulate KV-cache states. If you are batching or running multiple independent documents sequentially in the same memory context, you must manually clear the KV-cache between documents to prevent semantic context bleeding.
// 1. Clear KV cache for isolated document processing
self.model.clear_kv_cache();
// 2. Forward pass
let hidden_states = self.model.forward_hidden_states(&tokens)?;
// 3. Last-Token Pooling
let seq_len = hidden_states.dim(1)?;
let pool = hidden_states.narrow(1, seq_len - 1, 1)?.flatten_all()?;
let vec = pool.to_vec1::<f32>()?;
// 4. L2 Normalization
let norm = vec.iter().map(|x| x * x).sum::<f32>().sqrt();
let normalized = vec.iter().map(|x| x / norm).collect::<Vec<f32>>();
Refer to the original Microsoft Harrier-oss-v1 license.