AMALIA-9B-0626-SFT-GGUF

<div style="border:2px solid currentColor; font-family:ui-monospace,'SF Mono','Cascadia Mono',Consolas,monospace;"> <div style="border-bottom:1px solid currentColor; padding:6px 12px; font-size:11px; letter-spacing:3px; text-transform:uppercase; opacity:0.7; text-align:center;">KIKOCIS // EUROPEAN-PORTUGUESE LLM // IMATRIX GGUF + KLD</div> <div style="padding:14px; display:flex; flex-wrap:wrap; align-items:center; justify-content:center; gap:18px;"> <pre style="margin:0; font-size:9px; line-height:1.15;"> ╔═══════════════╗ ║ A M A L I A ║ pt-PT ╚═══════════════╝ ┌──┐ ┌──┐ ┌──┐ ┌──┐ │Q3│ │Q4│ │Q5│ │Q8│ ← ladder └──┘ └──┘ └──┘ └──┘ ●────●────●────● KLD vs F16 </pre> <div style="text-align:center;"> <div style="font-size:23px; font-weight:800; letter-spacing:1px;">AMALIA-9B · GGUF</div> <div style="font-size:12.5px; letter-spacing:1px; opacity:0.8; margin-top:5px;"><span style="white-space:nowrap;">llama · 9B</span> · <span style="white-space:nowrap;">32K ctx</span> · <span style="white-space:nowrap;">imatrix (pt)</span> · <span style="white-space:nowrap;">KLD-measured</span></div> </div> </div> <table style="display:table; table-layout:fixed; width:100%; margin:0; border-collapse:collapse; font-size:12px;"> <tr> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">FORMAT</div><div style="font-weight:700;">GGUF (imatrix)</div></td> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">SIZES</div><div style="font-weight:700;">3.1 – 9.1 GB</div></td> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">ARCH</div><div style="font-weight:700;">Llama · 42L</div></td> <td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">CONTEXT</div><div style="font-weight:700;">32768</div></td> </tr> <tr> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">IMATRIX</div><div style="font-weight:700;">pt-PT corpus</div></td> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">VALIDATION</div><div style="font-weight:700;">KLD vs F16</div></td> <td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">LANGUAGE</div><div style="font-weight:700;">Português (EU)</div></td> <td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">LICENSE</div><div style="font-weight:700;">Apache-2.0</div></td> </tr> </table> </div>

AMALIA-9B-0626-SFT — GGUF (imatrix + KLD)

imatrix-quantized GGUFs of AMALIA-9B, an open European-Portuguese instruction model — one of the first open LLMs developed in Portugal. Runs from ~3.1 GB (IQ2_M) to ~9.1 GB (Q8_0). Calibrated (imatrix) on Portuguese text, and every quant is measured against the full-precision model with KLD so you can pick a size with the fidelity cost in front of you. Credit: this is the AMALIA-LLM project's model — amalia-llm/AMALIA-9B-0626-SFT; ours is the quant ladder + metrics.

✅ Recommended files

📦 Files (the full ladder)

<!-- K-quants (Q*_K) = solid general quants. I-quants (IQ*) = smaller, need an imatrix (shipped). Higher number = more bits = closer to the original + bigger. -->

📊 Metrics — objective quality vs the F16 reference

KLD (Kullback–Leibler divergence, nats) measures how far each quant's output distribution drifts from the full-precision model — the gold-standard fidelity metric (lower = closer). Top-1 match = how often the quant's top token agrees with F16. Measured with llama-perplexity --kl-divergence over a Portuguese corpus at ctx 2048, vs the F16 GGUF.

_{Full per-quant reports in reports/; machine-readable summary in metrics/quant-summary.csv; SHA-256 of every file in reports/artifact-sha256sums.txt. Note: this is an instruction-tuned model measured on raw text, so the absolute PPL is higher than a base model's — what matters here is the relative KLD/top-1 across quants.}

Why does Q5_K_M beat Q6_K on KLD here? It's real and reproducible for this model: Q5_K_M's imatrix-weighted bit allocation is very effective, so it lands closer to F16 on mean-KLD than Q6_K (which stays marginally ahead on top-1). Both are near-lossless — so Q5_K_M is the better value, and Q8_0 is the true fidelity ceiling.

A note on i-quants: we tested the i-quants too. For this model the K-quants win at low bit — e.g. IQ3_M came out worse than Q3_K_M on KLD, so it isn't shipped. IQ2_M is included as the smallest option for very tight RAM, but with an honest quality cost (top-1 61%); if you can spare the space, Q3_K_M is a big step up.

📈 Charts

🧮 Will it fit? (RAM/VRAM cheat-sheet)

Memory ≈ weights + KV-cache (KV grows with context). Rough guide:

🚀 How to run it

# ollama (pulls straight from this repo)
ollama run hf.co/KikoCis/AMALIA-9B-0626-SFT-GGUF:Q5_K_M

# llama.cpp
llama-server -m amalia-9b-Q5_K_M.gguf -c 32768 --jinja
llama-cli   -m amalia-9b-Q5_K_M.gguf -p "Explica em português o que é a saudade." -ngl 99

Recommended sampling: temperature ~0.7, top_p ~0.9. It's a chat/instruct model (uses its built-in chat template) — prompt it in European Portuguese for best results.

⚠️ Good to know

• Strengths: European-Portuguese fluency and instruction-following; one of the first open LLMs built in Portugal.
• Limits: specialised for Portuguese — not a coding/agentic model, and not tuned for English. Low-bit quants (IQ2/Q3) trade real fidelity for size (see KLD).
• Absolute perplexity looks high only because it's measured on raw text with an instruct model; the KLD table is the fair quant-quality signal.

📊 Evaluation methodology (how the numbers were measured)

• What: quantization fidelity vs the F16 GGUF — not a task benchmark. llama-perplexity --kl-divergence (KLD mean/p95/max, ΔPPL, top-1 agreement).
• Corpus: European-Portuguese text (Wikipedia + diverse), ctx 2048, same corpus family used for the imatrix calibration.
• Reference: the model's own F16 GGUF (KLD = 0 by definition).
• Date: 2026-07. Caveat: relative fidelity ranking across quants of this model; not comparable across different models/corpora.

🔁 Provenance & reproducibility

• Scripts: scripts/ — exact convert → imatrix → quant → KLD commands.
• imatrix: amalia-9b-pt.imatrix — importance matrix (Portuguese calibration) used for the IQ/K quants.
• Checksums: reports/artifact-sha256sums.txt — shasum -a 256 -c to verify.
• Source: amalia-llm/AMALIA-9B-0626-SFT — weights unmodified (faithful quantization).

🗒️ Changelog

• 2026-07 v1: imatrix GGUF ladder (IQ2_M → Q8_0) + KLD/PPL metrics, Portuguese-calibrated.

📚 Credit & license

Model, weights, training data: © the AMALIA-LLM project — model · website · paper (PROPOR 2026). Quant ladder + imatrix (pt) + KLD/PPL metrics: KikoCis. Apache-2.0 (same as upstream). No weights modified.