Downloads · 30 days
763
15% of all-time downloads
Bonsai-27B-AWQ-4bit
prism-ml/Bonsai-27B-AWQ-4bit
Bonsai-27B-AWQ-4bit is a image-text-to-text model from prism-ml. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
All-time downloads
5.1K
Public
Parameters
27.4B
18.7 GB on disk
Likes
14
Trending 1
Repo makeup
Click a slice to open those files.
.safetensors18.7 GB · 100%
Parameter types
How the weights are stored.
I3224.3B · 89%
Try a prompt
Model card
From the Hugging Face model README
Bonsai-27B — AWQ 4-bit
Serve (sglang)
pip install sglang
python -m sglang.launch_server \
--model prism-ml/Bonsai-27B-AWQ-4bit \
--port 8000 \
--dtype bfloat16
Use
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"prism-ml/Bonsai-27B-AWQ-4bit","messages":[{"role":"user","content":"Who are you?"}],"max_tokens":100}'
Multi-GPU (8× H100)
# TP=2 DP=4 — 4 replicas, each split across 2 GPUs (throughput sweet spot)
python -m sglang.launch_server \
--model prism-ml/Bonsai-27B-AWQ-4bit \
--tp-size 2 --dp-size 4 \
--load-balance-method total_tokens \
--port 8000 --dtype bfloat16
- Task
- Image-Text-to-Text
- Library
- transformers
- Type
- qwen3_5
- License
- apache-2.0
- Created
- Jul 11, 2026
- Updated
- Jul 14, 2026