Downloads · 30 days
71
9% of all-time downloads
ALIA-40b-instruct-2601-GGUF
KikoCis/ALIA-40b-instruct-2601-GGUF
ALIA-40b-instruct-2601-GGUF is a text generation model from KikoCis. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as apache-2.0.
All-time downloads
810
Public
Repo size
131 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf131 GB · 100%
At a glance
- Task
- Text Generation
- Library
- llama.cpp
- License
- apache-2.0
- Access
- Public
- Created
- Jul 7, 2026
- Updated
- Jul 7, 2026
- SHA
- 1ec49c71
Base models
Model card
From the Hugging Face model README
ALIA-40b-instruct-2601 — GGUF (imatrix + KLD)
imatrix-quantized GGUFs of ALIA-40b, the Barcelona Supercomputing Center's ~46B sovereign LLM for Spain / the EU (Spanish + Catalan, Galician, Basque + European languages), with a 160K native context. Runs from ~13.5 GB (IQ2_M) to ~40 GB (Q8_0). Calibrated (imatrix) on Spanish text, with per-quant KLD fidelity vs the near-lossless Q8_0 reference. Credit: this is BSC-LT's model —
BSC-LT/ALIA-40b-instruct-2601; ours is the quant ladder + metrics.
✅ Recommended files
| Use case | File | Notes |
|---|---|---|
| Safe default | ALIA-40b-instruct-2601-Q4_K_M.gguf | The one most people want — good quality, ~23 GB (48 GB RAM). |
| Strong quality/size | ALIA-40b-instruct-2601-Q5_K_M.gguf | Closer to the reference, ~27 GB. |
| Runs on 32 GB | ALIA-40b-instruct-2601-Q3_K_M.gguf | ~19 GB — fits a 32 GB machine. |
| Smallest (tight RAM) | ALIA-40b-instruct-2601-IQ2_M.gguf | ~13.5 GB i-quant — runs a 46B on 24–32 GB, at a real quality cost. |
| Reference / max fidelity | ALIA-40b-instruct-2601-Q8_0.gguf | ~40 GB, near-lossless (the KLD reference). |
📦 Files (the ladder)
| Quant | Bits | File size | RAM (approx) | Notes |
|---|---|---|---|---|
| IQ2_M | ~2.7 | ~13.5 GB | 24–32 GB | Smallest — aggressive i-quant. |
| Q3_K_M | 3 | ~18.7 GB | 32 GB | Fits a 32 GB machine. |
| Q4_K_M | 4 | ~22.9 GB | 48 GB | Safe default. |
| Q5_K_M | 5 | ~26.8 GB | 48–64 GB | Strong quality/size. |
| Q8_0 | 8 | ~40 GB | 64 GB+ | Reference, near-lossless. |
📊 Metrics — fidelity vs the Q8_0 reference
KLD (Kullback–Leibler divergence, nats) measures how far each quant's output distribution drifts from the reference — lower = closer. Top-1 match = how often the quant's top token agrees with the reference. Measured with llama-perplexity --kl-divergence over a Spanish corpus at ctx 512.
Why the reference is Q8_0, not F16? ALIA-40b's F16 is ~81 GB and does not fit this machine's GPU (Metal). Q8_0 is near-lossless (its own KLD vs F16 would be ~0.005), so it's a faithful stand-in reference for measuring how much the smaller quants drift. KLD values here are therefore relative to Q8_0 (Q8_0 = 0 by definition).
| Model | Size GB | KLD mean | KLD p95 | KLD max | Top-1 match |
|---|---|---|---|---|---|
| Q8_0 (reference) | 40.0 | 0.0000 | 0.0000 | 0.0000 | 100.0% |
| Q5_K_M | 26.78 | 0.0098 | 0.0408 | 4.893 | 96.61% |
| Q4_K_M | 22.90 | 0.0337 | 0.1505 | 4.105 | 93.94% |
| Q3_K_M | 18.67 | 0.1114 | 0.5419 | 7.904 | 88.89% |
| IQ2_M | 13.54 | 0.3799 | 1.9178 | 12.259 | 77.82% |
<sub>Full per-quant reports in reports/; machine-readable summary in metrics/quant-summary.csv; SHA-256 of every file in reports/artifact-sha256sums.txt.</sub>
📈 Charts
🧮 Will it fit? (RAM cheat-sheet)
A 46B is memory-hungry; add KV-cache on top (it grows with context — 160K is a lot).
| you have | quant | context |
|---|---|---|
| 24 GB | IQ2_M | ~8–16K |
| 32 GB | Q3_K_M / IQ2_M | ~16–32K |
| 48 GB | Q4_K_M / Q5_K_M | ~32–64K |
| 64 GB+ | Q8_0 | large (up to 160K with room) |
🚀 How to run it
# ollama
ollama run hf.co/KikoCis/ALIA-40b-instruct-2601-GGUF:Q4_K_M
# llama.cpp
llama-server -m ALIA-40b-instruct-2601-Q4_K_M.gguf -c 32768 --jinja -ngl 99
Recommended sampling: temperature ~0.7, top_p ~0.9. Chat/instruct model (uses its built-in template) — great for Spanish and the co-official languages (Catalan, Galician, Basque) + European languages.
⚠️ Good to know
- Strengths: a genuinely sovereign, EU-built 46B — strong Spanish + co-official + European multilinguality, 160K context, permissive licence.
- Limits: it's a 46B — even the smallest quant needs ~24 GB RAM; IQ2_M trades real quality for size. Not a specialised coding model.
- KLD is measured vs Q8_0 (F16 doesn't fit this GPU) — see the note above.
📊 Evaluation methodology
- What: quantization fidelity vs the Q8_0 reference —
llama-perplexity --kl-divergence(KLD mean/p95/max, ΔPPL, top-1 agreement). - Corpus: Spanish text, ctx 512, same corpus used for imatrix calibration.
- Reference: Q8_0 (near-lossless stand-in for F16, which is too large for this GPU).
- Date: 2026-07. Caveat: relative fidelity ranking across quants of this model.
🔁 Provenance & reproducibility
- Scripts:
scripts/— exact convert → Q8 → imatrix → quant → KLD commands. - imatrix:
alia-40b-es.imatrix— importance matrix (Spanish calibration). - Checksums:
reports/artifact-sha256sums.txt. - Source:
BSC-LT/ALIA-40b-instruct-2601— weights unmodified (faithful quantization).
🗒️ Changelog
- 2026-07 v1: initial imatrix GGUF ladder (IQ2_M → Q8_0) + KLD metrics, Spanish-calibrated.
📚 Credit & license
Model, weights, training: © Barcelona Supercomputing Center — the ALIA / langtech-bsc project (model · ALIA-kit). Quant ladder + imatrix (es) + KLD metrics: KikoCis. Apache-2.0 (same as upstream). No weights modified.
- Task
- Text Generation
- Library
- llama.cpp
- License
- apache-2.0
- Languages
- es, ca, gl, eu, en
- Created
- Jul 7, 2026
- Updated
- Jul 7, 2026