Downloads · 30 days
18
100% of all-time downloads
maiui_8b_llamacpp_quantized
smehdia/maiui_8b_llamacpp_quantized
maiui_8b_llamacpp_quantized is a machine learning model from smehdia. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
OpenAI-compatible vision API for MAI-UI-8B (Q4KM + mmproj).
All-time downloads
18
Public
Repo size
5.8 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf5.8 GB · 97%
At a glance
- Access
- Public
- Created
- Sep 21, 2026
- Updated
- Sep 21, 2026
- SHA
- 56c9691b
Model card
From the Hugging Face model README
MAI-UI-8B Server Deployment Package
OpenAI-compatible vision API for MAI-UI-8B (Q4_K_M + mmproj).
Contents
| Path | Purpose |
|---|---|
bin/llama-server | CUDA-built inference server |
bin/lib*.so* | Required shared libraries |
models/*.gguf | Quantized weights + vision projector |
start_server.sh | Launch script |
Requirements (target machine)
- Linux x86_64
- NVIDIA GPU with CUDA 12+ drivers (built for RTX 50-series / sm_120)
- ~11 GB VRAM for inference (16 GB GPU recommended)
Start
chmod +x start_server.sh
CUDA_VISIBLE_DEVICES=0 ./start_server.sh
API: http://localhost:8080/v1
Verify
curl http://localhost:8080/v1/models
Client
Point agents at http://<host>:8080/v1 with model_name matching the served GGUF name.
- Created
- Sep 21, 2026
- Updated
- Sep 21, 2026