Downloads · 30 days
308
14% of all-time downloads
PLLuM-8x7B-nc-instruct-GGUF
Nondzu/PLLuM-8x7B-nc-instruct-GGUF
PLLuM-8x7B-nc-instruct-GGUF is a machine learning model from Nondzu. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc-by-nc-4.0.
DISCLAIMER: This is state of the art quantized model. I am not the author of the original model. I am only hosting the quantized models. I do not take any responsibility for the models.
All-time downloads
2.3K
Public
Repo size
293 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.gguf293 GB · 100%
At a glance
- License
- cc-by-nc-4.0
- Access
- Public
- Created
- Feb 24, 2025
- Updated
- Feb 25, 2025
- SHA
- 08a27b28
Base models
Model card
From the Hugging Face model README
PLLuM-8x7B-nc-instruct GGUF Quantizations by Nondzu
DISCLAIMER: This is state of the art quantized model. I am not the author of the original model. I am only hosting the quantized models. I do not take any responsibility for the models.
This repository contains GGUF quantized versions of the PLLuM-8x7B-nc-instruct model. All quantizations were performed using the llama.cpp (release b4768). These quantized models can be run in LM Studio or any other llama.cpp–based project.
Prompt Format
Use the following prompt structure:
???
Available Files
Below is a list of available quantized model files along with their quantization type, file size, and a short description.
| Filename | Quant Type | File Size | Description |
|---|---|---|---|
| PLLuM-8x7B-nc-instruct-Q2_K.gguf | Q2_K | 17 GB | Very low quality but surprisingly usable. |
| PLLuM-8x7B-nc-instruct-Q3_K.gguf | Q3_K | 21 GB | Low quality, suitable for setups with very limited RAM. |
| PLLuM-8x7B-nc-instruct-Q3_K_L.gguf | Q3_K_L | 23 GB | High quality; recommended for quality-focused usage. |
| PLLuM-8x7B-nc-instruct-Q3_K_M.gguf | Q3_K_M | 21 GB | Very high quality, near perfect output – recommended. |
| PLLuM-8x7B-nc-instruct-Q3_K_S.gguf | Q3_K_S | 20 GB | Moderate quality with improved space efficiency. |
| PLLuM-8x7B-nc-instruct-Q4_K_M.gguf | Q4_K_M | 27 GB | Default quality for most use cases – recommended. |
| PLLuM-8x7B-nc-instruct-Q4_K_S.gguf | Q4_K_S | 25 GB | Slightly lower quality with enhanced space savings – recommended when size is a priority. |
| PLLuM-8x7B-nc-instruct-Q5_K_M.gguf | Q5_K_M | 31 GB | High quality – recommended. |
| PLLuM-8x7B-nc-instruct-Q5_K_S.gguf | Q5_K_S | 31 GB | High quality, offered as an alternative with minimal quality loss. |
| PLLuM-8x7B-nc-instruct-Q6_K.gguf | Q6_K | 36 GB | Very high quality with quantized embed/output weights. |
| PLLuM-8x7B-nc-instruct-Q8_0.gguf | Q8_0 | 47 GB | Maximum quality quantization. |
Downloading Using Hugging Face CLI
<details> <summary>Click to view download instructions</summary>First, ensure you have the Hugging Face CLI installed:
pip install -U "huggingface_hub[cli]"
Then, target a specific file to download:
huggingface-cli download Nondzu/PLLuM-8x7B-instruct-nc-GGUF --include "PLLuM-8x7B-instruct-nc-Q4_K_M.gguf" --local-dir ./
For larger files, you can specify a new local directory (e.g., PLLuM-8x7B-instruct-nc-Q8_0) or download them directly into the current directory (./).
- License
- cc-by-nc-4.0
- Languages
- pl
- Created
- Feb 24, 2025
- Updated
- Feb 25, 2025