Downloads · 30 days
126
5% of all-time downloads
Mellum2-12B-A2.5B-Thinking-GGUF
CodeFault/Mellum2-12B-A2.5B-Thinking-GGUF
Mellum2-12B-A2.5B-Thinking-GGUF is a text generation model from CodeFault. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
Quantized GGUF version of JetBrains/Mellum2-12B-A2.5B-Thinking. These were generated using the default settings with llama-quantize (b9482).
All-time downloads
2.4K
Public
Repo size
62.3 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.gguf55.4 GB · 100%
At a glance
- Task
- Text Generation
- License
- apache-2.0
- Access
- Public
- Created
- Jun 2, 2026
- Updated
- Jun 4, 2026
- SHA
- d64ed4d4
Try a prompt
Base models
Model card
From the Hugging Face model README
Mellum2-12B-A2.5B-Thinking - GGUF
Quantized GGUF version of JetBrains/Mellum2-12B-A2.5B-Thinking.
These were generated using the default settings with llama-quantize (b9482).
Quantizations provided
| File | Quantization | Size |
|---|---|---|
Mellum2-12B-A2.5B-Thinking-Q4_0.gguf | Q4_0<sup>1</sup> | 6.91 GB |
Mellum2-12B-A2.5B-Thinking-Q4_K_S.gguf | Q4_K_S | 7.4 GB |
Mellum2-12B-A2.5B-Thinking-Q4_K_M.gguf | Q4_K_M | 8.07 GB |
Mellum2-12B-A2.5B-Thinking-Q5_K_M.gguf | Q5_K_M | 9.21 GB |
Mellum2-12B-A2.5B-Thinking-Q6_K.gguf | Q6_K | 10.9 GB |
Mellum2-12B-A2.5B-Thinking-Q8_0.gguf | Q8_0 | 12.9 GB |
<small>1: Q4_0 is not recommended. Perplexity doubled which suggests degredated quality, and I encountered endlessly repeating tokens with my test prompt.</small>
Perplexity test
I tested perplexity using llama-perplexity and Salesforce's wikitext-2-raw-v1.
| File | Quantization | Ctx | PPL |
|---|---|---|---|
Mellum2-12B-A2.5B-Thinking-Q4_0.gguf | Q4_0 | 512 | 18.6689 +/- 0.17737 |
Mellum2-12B-A2.5B-Thinking-Q4_K_S.gguf | Q4_K_S | 512 | 9.8269 +/- 0.07248 |
Mellum2-12B-A2.5B-Thinking-Q4_K_M.gguf | Q4_K_M | 512 | 9.7410 +/- 0.07128 |
Mellum2-12B-A2.5B-Thinking-Q5_K_M.gguf | Q5_K_M | 512 | 9.4490 +/- 0.06807 |
Mellum2-12B-A2.5B-Thinking-Q6_K.gguf | Q6_K | 512 | 9.7329 +/- 0.07207 |
Mellum2-12B-A2.5B-Thinking-Q8_0.gguf | Q8_0 | 512 | 9.3657 +/- 0.06734 |
Mellum2-12B-A2.5B-Thinking-BF16.gguf | BF16 | 512 | 9.4037 +/- 0.06784 |
Serving with llama.cpp
llama.cpp added support for Mellum2 in release b9482. It has a max context size of 131,072. This can be served using:
llama-server \
-hf CodeFault/Mellum2-12B-A2.5B-Thinking-GGUF:Q5_K_M \
--temp 0.6 \
--top-p 0.95 \
--top-k 20
- Task
- Text Generation
- License
- apache-2.0
- Languages
- en
- Created
- Jun 2, 2026
- Updated
- Jun 4, 2026