Downloads · 30 days
142
6% of all-time downloads
Mellum2-12B-A2.5B-Instruct-GGUF
CodeFault/Mellum2-12B-A2.5B-Instruct-GGUF
Mellum2-12B-A2.5B-Instruct-GGUF is a text generation model from CodeFault. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
Quantized GGUF version of JetBrains/Mellum2-12B-A2.5B-Instruct. These were generated using the default settings with llama-quantize (b9482).
All-time downloads
2.5K
Public
Repo size
62.3 GB
Likes
4
Public
Repo makeup
Click a slice to open those files.
.gguf55.4 GB · 100%
At a glance
- Task
- Text Generation
- License
- apache-2.0
- Access
- Public
- Created
- Jun 2, 2026
- Updated
- Jun 4, 2026
- SHA
- 33c8fbfc
Try a prompt
Base models
Model card
From the Hugging Face model README
Mellum2-12B-A2.5B-Instruct - GGUF
Quantized GGUF version of JetBrains/Mellum2-12B-A2.5B-Instruct.
These were generated using the default settings with llama-quantize (b9482).
Quantizations provided
| File | Quantization | Size |
|---|---|---|
Mellum2-12B-A2.5B-Instruct-Q4_0.gguf | Q4_0<sup>1</sub> | 6.91 GB |
Mellum2-12B-A2.5B-Instruct-Q4_K_S.gguf | Q4_K_S | 7.4 GB |
Mellum2-12B-A2.5B-Instruct-Q4_K_M.gguf | Q4_K_M | 8.07 GB |
Mellum2-12B-A2.5B-Instruct-Q5_K_M.gguf | Q5_K_M | 9.21 GB |
Mellum2-12B-A2.5B-Instruct-Q6_K.gguf | Q6_K | 10.9 GB |
Mellum2-12B-A2.5B-Instruct-Q8_0.gguf | Q8_0 | 12.9 GB |
<small>1: Q4_0 is not recommended. Perplexity increased significantly which suggests degredated quality. I did not encounter endlessly repeating tokens like with the thinking variation at Q4_0.</small>
Perplexity test
I tested perplexity using llama-perplexity and Salesforce's wikitext-2-raw-v1.
| File | Quantization | Ctx | PPL |
|---|---|---|---|
Mellum2-12B-A2.5B-Instruct-Q4_0.gguf | Q4_0 | 512 | 180.4777 +/- 3.04078 |
Mellum2-12B-A2.5B-Instruct-Q4_K_S.gguf | Q4_K_S | 512 | 14.1798 +/- 0.13170 |
Mellum2-12B-A2.5B-Instruct-Q4_K_M.gguf | Q4_K_M | 512 | 14.0943 +/- 0.13090 |
Mellum2-12B-A2.5B-Instruct-Q5_K_M.gguf | Q5_K_M | 512 | 14.6015 +/- 0.13815 |
Mellum2-12B-A2.5B-Instruct-Q6_K.gguf | Q6_K | 512 | 17.7939 +/- 0.18152 |
Mellum2-12B-A2.5B-Instruct-Q8_0.gguf | Q8_0 | 512 | 13.2165 +/- 0.12008 |
Mellum2-12B-A2.5B-Instruct-BF16.gguf | BF16 | 512 | 14.7864 +/- 0.14076 |
Serving with llama.cpp
llama.cpp added support for Mellum2 in release b9482. It has a max context size of 131,072. This can be served using:
llama-server \
-hf CodeFault/Mellum2-12B-A2.5B-Instruct-GGUF:Q5_K_M \
--temp 0.6 \
--top-p 0.95 \
--top-k 20
- Task
- Text Generation
- License
- apache-2.0
- Languages
- en
- Created
- Jun 2, 2026
- Updated
- Jun 4, 2026