Downloads · 30 days
1.4K
5% of all-time downloads
mamba-gguf
devingulliver/mamba-gguf
mamba-gguf is a text generation model from devingulliver. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
These are the Mamba base models, converted to GGUF for use with llama.cpp, in a variety of precisions (2, 3, 4, 5, 6, 8, 16, and 32-bit).
All-time downloads
26.4K
Public
Repo size
126 GB
Likes
3
Public
Repo makeup
Click a slice to open those files.
.gguf126 GB · 100%
At a glance
- Task
- Text Generation
- License
- apache-2.0
- Access
- Public
- Created
- Mar 12, 2024
- Updated
- Mar 12, 2024
- SHA
- 4f5fc95b
Try a prompt
Base models
Model card
From the Hugging Face model README
Mamba GGUF
These are the Mamba base models, converted to GGUF for use with llama.cpp, in a variety of precisions (2, 3, 4, 5, 6, 8, 16, and 32-bit).
Please click "Files and versions" at the top of the page to choose your desired model size, and then click the "📦LFS ↓" button next to your desired quantization.
Here is a table adapted from TheBloke explaining the various precisions:
| Quant method | Use case |
|---|---|
| Q2_K | significant quality loss - not recommended for most purposes |
| Q3_K_S | very small, high quality loss |
| Q3_K_M | very small, high quality loss |
| Q3_K_L | small, substantial quality loss |
| Q4_0 | legacy; small, very high quality loss - prefer using Q3_K_M |
| Q4_K_S | small, greater quality loss |
| Q4_K_M | medium, balanced quality - recommended |
| Q5_0 | legacy; medium, balanced quality - prefer using Q4_K_M |
| Q5_K_S | large, low quality loss - recommended |
| Q5_K_M | large, very low quality loss - recommended |
| Q6_K | very large, extremely low quality loss |
| Q8_0 | very large, extremely low quality loss - not recommended |
| F16 | half precision - almost identical to the original |
| F32 | original precision - recommended by the Mamba authors |
- Task
- Text Generation
- License
- apache-2.0
- Created
- Mar 12, 2024
- Updated
- Mar 12, 2024