Downloads · 30 days
24
18% of all-time downloads
marcinm1234/Llama-PLLuM-70B-instruct-2512-Q8_0-GGUF
Llama-PLLuM-70B-instruct-2512-Q8_0-GGUF is a text generation model from marcinm1234. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as llama3.1.
GGUF quantization of CYFRAGOVPL/Llama-PLLuM-70B-instruct-2512.
Downloads · 30 days
24
18% of all-time downloads
All-time downloads
137
Public
Repo size
75 GB
Likes
0
Public
Click a slice to open those files.
.gguf75 GB · 100%
From the Hugging Face model README
GGUF quantization of CYFRAGOVPL/Llama-PLLuM-70B-instruct-2512.
Built with Llama. This repository contains only a GGUF quantization of the model — I did not train, fine-tune, or otherwise create the underlying model. All credit for the model belongs to the PLLuM consortium / HIVE AI (CYFRAGOVPL), who trained it on ~150B tokens of Polish text using the Bem2 HPC cluster. This upload exists solely to provide a
llama.cpp-compatible GGUF, which was not otherwise available for theinstruct-2512variant at the time of upload.
instruct (instruction-following; not the preference-aligned chat variant)LlamaForCausalLM — 80 layers, hidden 8192, 64 attn heads / 8 KV heads (GQA), 131072 context, rope theta 500000| Quant | Size | Notes |
|---|---|---|
| Q8_0 | ~75 GB | Highest practical fidelity; round-to-nearest, no imatrix |
Converted with llama.cpp:
python convert_hf_to_gguf.py ./Llama-PLLuM-70B-instruct-2512 \
--outfile Llama-PLLuM-70B-instruct-2512-Q8_0.gguf \
--outtype q8_0
Despite the Llama base, this model does NOT use the standard Llama-3.1
header format (<|start_header_id|>...<|eot_id|>). It uses a Mistral-style
[INST] ... [/INST] template, embedded in the GGUF metadata. A system message
is folded into the first [INST] block, separated from the user content by
\n\n:
<|begin_of_text|>[INST]{system}\n\n{user}[/INST]{assistant}<|end_of_text|>
<|begin_of_text|> (128000) · EOS: <|end_of_text|> (128001)llama-server detects and applies this template automatically from GGUF metadata.
If you bypass it, replicate the [INST] format above (do not use llama3 template).# server
llama-server -m Llama-PLLuM-70B-instruct-2512-Q8_0.gguf -c 16384 -ngl 99 --port 8080
# OpenAI-compatible request (system folds into first [INST] automatically)
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
"messages": [
{"role": "system", "content": "Jesteś pomocnym asystentem."},
{"role": "user", "content": "Napisz krótkie podsumowanie."}
],
"temperature": 0.1
}'
This is not a reasoning/thinking model (thinking = 0); for deterministic
tasks (extraction, correction) low temperature (0.0–0.2) is recommended.
This GGUF is a derivative of a Llama-3.1-based model and is distributed under the Llama 3.1 Community License, in addition to any license terms set by the original authors (PLLuM / CYFRAGOVPL) on the base model card. Please review and comply with both before use. "Llama" is included in the model name in accordance with the Llama 3.1 license requirements for derivative models.