Downloads · 30 days
42
12% of all-time downloads
InfinityKumon-2x7B-GGUF
R136a1/InfinityKumon-2x7B-GGUF
InfinityKumon-2x7B-GGUF is a machine learning model from R136a1. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
All-time downloads
352
Public
Repo size
76.9 GB
Likes
6
Public
Repo makeup
Click a slice to open those files.
.gguf51.2 GB · 67%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Mar 26, 2024
- Updated
- Mar 27, 2024
- SHA
- 0a3a4927
Model card
From the Hugging Face model README
InfinityKumon-2x7B
GGUF - Imatrix quant of InfinityKumon-2x7B
Another MoE merge from Endevor/InfinityRP-v1-7B and grimjim/kukulemon-7B.
The reason? Because I like InfinityRP-v1-7B so much and wondering if I can improve it even more by merging 2 great models into MoE.
Perplexity
Using llama.cpp/perplexity with private roleplay dataset.
| Format | PPL |
|---|---|
| FP16 | 3.1748 +/- 0.11928 |
| Q8_0 | 3.1734 +/- 0.11935 |
| Q6_K | 3.1752 +/- 0.11899 |
| Q5_K_M | 3.1731 +/- 0.11892 |
| IQ4_NL | 3.1752 +/- 0.11943 |
| IQ3_M | 3.1773 +/- 0.11528 |
| Q2_K | 3.2309 +/- 0.11996 |
I don't really recomend using Q2_K based on the ppl, the other quants are fine.
Prompt format:
Alpaca or ChatML
- License
- apache-2.0
- Languages
- en
- Created
- Mar 26, 2024
- Updated
- Mar 27, 2024