Downloads · 30 days
19
7% of all-time downloads
majentik/harrier-oss-v1-0.6b-GGUF-Q4_K_M
harrier-oss-v1-0.6b-GGUF-Q4_K_M is a feature extraction model from majentik. Use it when you need embeddings to search or compare text. It is set up for gguf. The card lists the license as mit.
[!TIP] KV-cache quantization (upstream, no fork needed): llama.cpp/Ollama cover this natively — -ctk q80 -ctv q80 (~half KV memory, negligible quality loss) or -ctk q40 -ctv q40 (~quarter memory, small quality cost).…
Downloads · 30 days
19
7% of all-time downloads
All-time downloads
271
Public
Repo size
397 MB
Likes
0
Public
Click a slice to open those files.
.gguf397 MB · 100%
From the Hugging Face model README
[!TIP] KV-cache quantization (upstream, no fork needed): llama.cpp/Ollama cover this natively —
-ctk q8_0 -ctv q8_0(~half KV memory, negligible quality loss) or-ctk q4_0 -ctv q4_0(~quarter memory, small quality cost). In Ollama:OLLAMA_KV_CACHE_TYPE=q8_0withOLLAMA_FLASH_ATTENTION=1.
llama.cpp GGUF Q4_K_M quantization of microsoft/harrier-oss-v1-0.6b.
llama-quantize (upstream llama.cpp)convert_hf_to_gguf.pyllama-embedding -m harrier-0.6b-Q4_K_M.gguf -p "What is Harrier-OSS?"
MIT — inherited from the upstream Harrier-OSS-v1-0.6B.