Downloads · 30 days
5.3K
43% of all-time downloads
Rioky/Mistral-Small-4-119B-2603-heretic-gguf
Mistral-Small-4-119B-2603-heretic-gguf is a image-text-to-text model from Rioky. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
- MoE Architecture: 128 experts with 4 active per token (6.5B activated parameters) - Multimodal: Accepts both text and image input, produces text output - Reasoning Mode: Toggle between fast instant reply and reasoni…
Downloads · 30 days
5.3K
43% of all-time downloads
All-time downloads
12.4K
Public
Repo size
1.1 TB
Likes
0
Public
Click a slice to open those files.
.gguf1.1 TB · 100%
From the Hugging Face model README
| Field | Value |
|---|---|
| Base Model | Mistral Small 4 119B 2603 by Mistralai |
| Heretic Model | Mistral Small 4 119B 2603 Heretic by darkc0de |
| Architecture | Mistral4 (MoE, MLA attention) |
| Total Parameters | 119B |
| Active Parameters | 6.5B per token |
| Experts | 128 experts, 4 active |
| Context Length | 256k tokens |
| Metric | Value |
|---|---|
| KL Divergence | 0.0167 (0 for refusable tokens by definition) |
| Refusals | 27/100 (98/100 original model) |
Standard quantizations without importance matrix (imatrix) optimization.
| File | Type | Size | BPW | Description |
|---|---|---|---|---|
Mistral-Small-4-119B-2603-heretic-BF16.gguf | BF16 | 221.6 GiB | 16.00 | Base model, highest quality |
Mistral-Small-4-119B-2603-heretic-Q8_0.gguf | Q8_0 | 117.8 GiB | 8.00 | Very high quality |
Mistral-Small-4-119B-2603-heretic-Q6_K.gguf | Q6_K | 91.0 GiB | 6.00 | High quality |
Mistral-Small-4-119B-2603-heretic-Q5_K_M.gguf | Q5_K_M | 78.5 GiB | 5.37 | Good quality/size balance |
Mistral-Small-4-119B-2603-heretic-Q5_K_S.gguf | Q5_K_S | 76.3 GiB | 5.25 | |
Mistral-Small-4-119B-2603-heretic-Q4_K_M.gguf | Q4_K_M | 67.2 GiB | 4.58 | Recommended |
Mistral-Small-4-119B-2603-heretic-Q4_K_S.gguf | Q4_K_S | 63.1 GiB | 4.37 | |
Mistral-Small-4-119B-2603-heretic-IQ4_NL.gguf | IQ4_NL | 63.0 GiB | 4.50 | Non-linear quantization |
Mistral-Small-4-119B-2603-heretic-Q3_K_L.gguf | Q3_K_L | 57.4 GiB | 3.70 | |
Mistral-Small-4-119B-2603-heretic-Q3_K_M.gguf | Q3_K_M | 53.1 GiB | 3.63 | |
Mistral-Small-4-119B-2603-heretic-Q3_K_S.gguf | Q3_K_S | 47.8 GiB | 3.41 | |
Mistral-Small-4-119B-2603-heretic-Q2_K.gguf | Q2_K | 40.4 GiB | 2.96 | Low memory usage |