Downloads · 30 days
158
5% of all-time downloads
PLLuM-8x7B-chat-GGUF
Nondzu/PLLuM-8x7B-chat-GGUF
PLLuM-8x7B-chat-GGUF is a machine learning model from Nondzu. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
DISCLAIMER: This is a quantized version of an existing model PLLuM-8x7B-chat. I am not the author of the original model. I am only hosting the quantized models. I do not take any responsibility for the models.
All-time downloads
3.3K
Public
Repo size
325 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf376 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Feb 24, 2025
- Updated
- Feb 24, 2025
- SHA
- c2f9838e
Base models
Model card
From the Hugging Face model README
PLLuM-8x7B-chat GGUF Quantizations by Nondzu
DISCLAIMER: This is a quantized version of an existing model PLLuM-8x7B-chat. I am not the author of the original model. I am only hosting the quantized models. I do not take any responsibility for the models.
Prompt Format
Use the following prompt structure:
???
Available Files
Below is a list of available quantized model files along with their quantization type, file size, and a short description.
| Filename | Quant Type | File Size | Description |
|---|---|---|---|
| PLLuM-8x7B-chat-Q2_K.gguf | Q2_K | 17 GB | Very low quality but surprisingly usable. |
| PLLuM-8x7B-chat-Q3_K.gguf | Q3_K | 21 GB | Low quality, suitable for setups with very limited RAM. |
| PLLuM-8x7B-chat-Q3_K_L.gguf | Q3_K_L | 23 GB | High quality; recommended for quality-focused usage. |
| PLLuM-8x7B-chat-Q3_K_M.gguf | Q3_K_M | 21 GB | Very high quality, near perfect output – recommended. |
| PLLuM-8x7B-chat-Q3_K_S.gguf | Q3_K_S | 20 GB | Moderate quality with improved space efficiency. |
| PLLuM-8x7B-chat-Q4_K.gguf | Q4_K | 27 GB | Good quality for standard use. |
| PLLuM-8x7B-chat-Q4_K_M.gguf | Q4_K_M | 27 GB | Default quality for most use cases – recommended. |
| PLLuM-8x7B-chat-Q4_K_S.gguf | Q4_K_S | 25 GB | Slightly lower quality with enhanced space savings – recommended when size is a priority. |
| PLLuM-8x7B-chat-Q5_0.gguf | Q5_0 | 31 GB | Extremely high quality – the maximum quant available. |
| PLLuM-8x7B-chat-Q5_K.gguf | Q5_K | 31 GB | Very high quality – recommended for demanding use cases. |
| PLLuM-8x7B-chat-Q5_K_M.gguf | Q5_K_M | 31 GB | High quality – recommended. |
| PLLuM-8x7B-chat-Q5_K_S.gguf | Q5_K_S | 31 GB | High quality, offered as an alternative with minimal quality loss. |
| PLLuM-8x7B-chat-Q6_K.gguf | Q6_K | 36 GB | Very high quality with quantized embed/output weights. |
| PLLuM-8x7B-chat-Q8_0.gguf | Q8_0 | 47 GB | Maximum quality quantization. |
Downloading Using Hugging Face CLI
<details> <summary>Click to view download instructions</summary>First, ensure you have the Hugging Face CLI installed:
pip install -U "huggingface_hub[cli]"
Then, target a specific file to download:
huggingface-cli download Nondzu/PLLuM-8x7B-chat-GGUF --include "PLLuM-8x7B-chat-Q4_K_M.gguf" --local-dir ./
For larger files, you can specify a new local directory (e.g., PLLuM-8x7B-chat-Q8_0) or download them directly into the current directory (./).
- License
- apache-2.0
- Languages
- pl
- Created
- Feb 24, 2025
- Updated
- Feb 24, 2025