Downloads · 30 days
11
2% of all-time downloads
majentik/harrier-oss-v1-0.6b-GGUF-IQ4_XS
harrier-oss-v1-0.6b-GGUF-IQ4_XS is a feature extraction model from majentik. Use it when you need embeddings to search or compare text. It is set up for gguf. The card lists the license as mit.
[!TIP] KV-cache quantization (upstream, no fork needed): llama.cpp/Ollama cover this natively — -ctk q80 -ctv q80 (~half KV memory, negligible quality loss) or -ctk q40 -ctv q40 (~quarter memory, small quality cost).…
Downloads · 30 days
11
2% of all-time downloads
All-time downloads
453
Public
Repo size
369 MB
Likes
0
Public
Click a slice to open those files.
.gguf369 MB · 100%
From the Hugging Face model README
[!TIP] KV-cache quantization (upstream, no fork needed): llama.cpp/Ollama cover this natively —
-ctk q8_0 -ctv q8_0(~half KV memory, negligible quality loss) or-ctk q4_0 -ctv q4_0(~quarter memory, small quality cost). In Ollama:OLLAMA_KV_CACHE_TYPE=q8_0withOLLAMA_FLASH_ATTENTION=1.
llama.cpp GGUF IQ4_XS quantization of microsoft/harrier-oss-v1-0.6b.
llama-quantize (upstream llama.cpp)convert_hf_to_gguf.pyllama-embedding -m harrier-0.6b-IQ4_XS.gguf -p "What is Harrier-OSS?"
MIT — inherited from the upstream Harrier-OSS-v1-0.6B.