Downloads · 30 days
35
2% of all-time downloads
OpenHermes-2.5-Mistral-7B-GGUF-ukrainian-imatrix
interrobang/OpenHermes-2.5-Mistral-7B-GGUF-ukrainian-imatrix
OpenHermes-2.5-Mistral-7B-GGUF-ukrainian-imatrix is a machine learning model from interrobang. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
A test quantization of OpenHermes-2.5-Mistral-7B by teknium using importance matrices computed on Ukrainian text, hopefully decreasing the coherence hit after quantization in Ukrainian at the cost of some performance…
All-time downloads
1.9K
Public
Repo size
24.8 GB
Likes
4
Public
Repo makeup
Click a slice to open those files.
.gguf21.9 GB · 88%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Feb 7, 2024
- Updated
- Feb 27, 2024
- SHA
- 18ac6ba7
Model card
From the Hugging Face model README
A test quantization of OpenHermes-2.5-Mistral-7B by teknium using importance matrices computed on Ukrainian text, hopefully decreasing the coherence hit after quantization in Ukrainian at the cost of some performance in other languages.
Importance matrix was computed in roughly 20 minutes with a Ryzen 5 3550H and GTX 1650 with 8 layers offloaded, with a context size of 512.
The calibration data is just a mix of my personal GPT chats, random words as well as random wikipedia articles, totaling about 15k-ish tokens, definitely not optimal, but it is in the repo for anyone to tinker with, as well as the computed imatrix
Will be updated with perplexity testing later, probably? 😭 Haven't done proper tests quite yet, feels better than old quants when chatting in Ukrainian, hopefully I get around to actually benching it somehow
- License
- apache-2.0
- Created
- Feb 7, 2024
- Updated
- Feb 27, 2024