Downloads · 30 days
534
7% of all-time downloads
Joseph717171/Hermes-3-Llama-3.1-8B-OQ8_0-F32.EF32.IQ4_K-Q8_0-GGUF
Hermes-3-Llama-3.1-8B-OQ8_0-F32.EF32.IQ4_K-Q8_0-GGUF is a machine learning model from Joseph717171. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Custom GGUF quants of NousResearch/Hermes-3-Llama-3.1-8B, where the Output Tensors are quantized to Q80 while the Embeddings are kept at F32. Enjoy! 🧠🔥🚀
Downloads · 30 days
534
7% of all-time downloads
All-time downloads
7.9K
Public
Repo size
322 GB
Likes
2
Public
Click a slice to open those files.
.gguf55.3 GB · 100%
From the Hugging Face model README
Custom GGUF quants of NousResearch/Hermes-3-Llama-3.1-8B, where the Output Tensors are quantized to Q8_0 while the Embeddings are kept at F32. Enjoy! 🧠🔥🚀
Update: This repo now contains OF32.EF32 GGUF IQuants for even more accuracy. Enjoy! 😋
UPDATE: This repo now contains updated O.E.IQuants, which were quantized, using a new F32-imatrix, using llama.cpp version: 4658 (855cd073). This particular version of llama.cpp added support for Non-Contiguous RMS Norms. This has enhanced model coherence and further increased model creativity (from testing).