Downloads · 30 days
263
5% of all-time downloads
batiai/Llama-4-Scout-17B-16E-Instruct-GGUF
Llama-4-Scout-17B-16E-Instruct-GGUF is a image-text-to-text model from batiai. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for llama.cpp. The card lists the license as other.
<p align="center" <a href="https://flow.bati.ai"<img src="https://img.shields.io/badge/BatiFlow-on--device%20AI-blue?style=for-the-badge&logo=apple" alt="BatiFlow"</a <a href="https://ollama.com/batiai/llama4-scout"<i…
Downloads · 30 days
263
5% of all-time downloads
All-time downloads
5.8K
Public
Repo size
245 GB
Likes
0
Public
Click a slice to open those files.
.gguf245 GB · 100%
From the Hugging Face model README
imatrix-calibrated GGUF quantizations of meta-llama/Llama-4-Scout-17B-16E-Instruct (109B total / 17B active MoE, 16 experts, multimodal). Quantized directly from official Meta BF16 weights by BatiAI.
mmproj (image-text-to-text)# Q4_K_M (recommended balance, 60GB, M4 Max 128GB ~ M2 Ultra 192GB)
ollama pull batiai/llama4-scout:q4
# IQ3_XXS (smallest, 38GB, M4 Max 64GB+)
ollama pull batiai/llama4-scout:iq3
# Q5_K_M (higher quality, 72GB, M2 Ultra 192GB+)
ollama pull batiai/llama4-scout:q5
| Quant | Size | Min RAM | Target Hardware | Notes |
|---|---|---|---|---|
| IQ3_XXS | 38GB | 48GB | M4 Max 64GB | aggressive 3-bit, imatrix-calibrated |
| Q3_K_M | 48GB | 56GB | M4 Max 64GB | balanced 3-bit |
| IQ4_XS | 52GB | 64GB | M4 Max 96GB | imatrix 4-bit (best size/quality) |
| Q4_K_M | 60GB | 72GB | M4 Max 128GB | recommended for most users |
| Q5_K_M | 72GB | 88GB | M2 Ultra 192GB | higher fidelity |
| Q6_K | 85GB | 96GB | M2 Ultra 192GB+ | near-original quality |
Mac note on
Q3_K_M: in every model we've benchmarked on Apple Silicon, Q3_K_M generated slower than Q4_K_M despite the smaller file — Granite 4.1 (+27%), Gemma 4 26B (+12%), Qwen3.8‑27B (+18%), Qwen3.6‑27B (+8%), on both M4 Max and M4 mini. Metal's Q3_K path is limited by dequantization compute rather than bandwidth. We have not measured this particular model's Q3/Q4 pair yet, so treat it as a strong prior, not a measurement: if Q4_K_M fits, take it. On CUDA the two are effectively tied, so this applies to Macs only.
Multimodal users: also download
mmproj-*-BF16.gguf(ormmproj-*-Q6_K.gguf) and use withllama-server --mmprojorllama-mtmd-cli.
| Your System | IQ3 (38GB) | Q4 (60GB) | Q5 (72GB) | Q6 (85GB) |
|---|---|---|---|---|
| Mac mini M4 16GB | ❌ | ❌ | ❌ | ❌ |
| Mac mini M4 Pro 24GB | ❌ | ❌ | ❌ | ❌ |
| Mac M4 Max 64GB | ✅ tight | ❌ | ❌ | ❌ |
| Mac M4 Max 96GB | ✅ | ✅ tight | ❌ | ❌ |
| Mac M4 Max 128GB | ✅ | ✅ | ✅ tight | ❌ |
| Mac M2 Ultra 192GB | ✅ | ✅ | ✅ | ✅ |
| Mac M3 Ultra 512GB | ✅ comfortable | ✅ | ✅ | ✅ |
ollama pull batiai/llama4-scout:q4
ollama run batiai/llama4-scout:q4
# Download GGUF + mmproj
hf download batiai/Llama-4-Scout-17B-16E-Instruct-GGUF \
--include "*Q4_K_M*" --include "mmproj-*-Q6_K.gguf" \
--local-dir ./llama4-scout
# Run with vision
llama-mtmd-cli \
-m ./llama4-scout/meta-llama-Llama-4-Scout-17B-16E-Instruct-Q4_K_M.gguf \
--mmproj ./llama4-scout/mmproj-meta-llama-Llama-4-Scout-17B-16E-Instruct-Q6_K.gguf \
--image input.jpg -p "Describe this image."
# Or as a server
llama-server -m ./llama4-scout/meta-llama-Llama-4-Scout-17B-16E-Instruct-Q4_K_M.gguf \
--mmproj ./llama4-scout/mmproj-meta-llama-Llama-4-Scout-17B-16E-Instruct-Q6_K.gguf \
-ngl 99 -c 32768 --port 8080
Llama4ForConditionalGeneration — 109B total / 17B active MoEAll GGUFs in this repo carry:
general.author = BatiAIgeneral.url = https://flow.bati.aiInherits Meta Llama 4 Community License. Commercial-friendly for organizations with < 700M MAU. See:
BatiFlow — free on-device AI automation for Mac.
<!-- BENCH-START -->Benchmarks coming once Mac M4 Max / M2 Ultra measurements complete.
<!-- BENCH-END -->