Llama-PLLuM-70B-chat-GGUF

Llama-PLLuM-70B-chat GGUF Quantizations by Nondzu

DISCLAIMER: This is a quantized version of an existing model Llama-PLLuM-70B-chat. I am not the author of the original model. I am only hosting the quantized models. I do not take any responsibility for the models.

This repository contains GGUF quantized versions of the Llama-PLLuM-70B-chat model. All quantizations were performed using the llama.cpp (release b4765). These quantized models can be run in LM Studio or any other llama.cpp–based project.

Prompt Format

Use the following prompt structure:

???

Available Files

Below is a list of available quantized model files along with their quantization type, file size, whether the file is split, and a short description.

*Files marked as "split" must be downloaded in full (all parts) to obtain the complete quantized model.

Downloading Using Hugging Face CLI

<details> <summary>Click to view download instructions</summary>

First, ensure you have the Hugging Face CLI installed:

pip install -U "huggingface_hub[cli]"

Then, target a specific file to download:

huggingface-cli download Nondzu/Llama-PLLuM-70B-chat-GGUF --include "Llama-PLLuM-70B-chat-Q4_K_M.gguf" --local-dir ./

For files larger than 50 GB that are split into multiple parts, use a wildcard to download all parts at once:

huggingface-cli download Nondzu/Llama-PLLuM-70B-chat-GGUF --include "Llama-PLLuM-70B-chat-Q8_0/*" --local-dir ./

You can specify a new local directory (e.g., Llama-PLLuM-70B-chat-Q8_0) or download them directly into the current directory (./).

</details>