Downloads · 30 days
14
4% of all-time downloads
Lumimaid-v0.2-12B-Q4_K_M-GGUF
mrcuddle/Lumimaid-v0.2-12B-Q4_K_M-GGUF
Lumimaid-v0.2-12B-Q4_K_M-GGUF is a text generation model from mrcuddle. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This model was converted to GGUF format from NeverSleep/Lumimaid-v0.2-12B using llama.cpp via Convert Model to GGUF.
All-time downloads
353
Public
Repo size
7.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf7.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Access
- Public
- Created
- Oct 16, 2024
- Updated
- Oct 20, 2024
- SHA
- 5968fc52
Try a prompt
Model card
From the Hugging Face model README
mrcuddle/Lumimaid-v0.2-12B-Q4_K_M-GGUF
This model was converted to GGUF format from NeverSleep/Lumimaid-v0.2-12B using llama.cpp via
Convert Model to GGUF.
Key Features:
- Quantized for reduced file size (GGUF format)
- Optimized for use with llama.cpp
- Compatible with llama-server for efficient serving
Refer to the original model card for more details on the base model.
Usage with llama.cpp
1. Install llama.cpp:
brew install llama.cpp # For macOS/Linux
2. Run Inference:
CLI:
llama-cli --hf-repo mrcuddle/Lumimaid-v0.2-12B-Q4_K_M-GGUF --hf-file lumimaid-v0.2-12b-q4_k_m.gguf -p "Your prompt here"
Server:
llama-server --hf-repo mrcuddle/Lumimaid-v0.2-12B-Q4_K_M-GGUF --hf-file lumimaid-v0.2-12b-q4_k_m.gguf -c 2048
For more advanced usage, refer to the llama.cpp repository.
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Created
- Oct 16, 2024
- Updated
- Oct 20, 2024