Downloads · 30 days
94
8% of all-time downloads
LiamVisionary/swarm-sovereign-scout-12b-GGUF
swarm-sovereign-scout-12b-GGUF is a image-text-to-text model from LiamVisionary. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for llama.cpp. The card lists the license as apache-2.0.
<p align="center" <img src="swarm-scout-card.jpg" alt="Swarm Scout mascot" width="520" </p
Downloads · 30 days
94
8% of all-time downloads
All-time downloads
1.2K
Public
Repo size
15.4 GB
Likes
1
Public
Click a slice to open those files.
.gguf8 GB · 100%
From the Hugging Face model README
Swarm Scout 12B is a low-refusal, coding/agentic GGUF build derived from
yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2, itself based on google/gemma-4-12B-it.
The upstream model is specialized for coding, terminal workflows, native Gemma 4 tool-use, multi-step agentic tasks, and reasoning. This release keeps that focus, then applies a local Heretic/abliteration pass targeted at reducing opening refusals while preserving behavior as much as possible.
Brand note: this release is now named Swarm Scout. The Hugging Face repo slug and GGUF filenames still contain swarm-sovereign-scout so existing download commands, Modal caches, and copied links keep working.
Want to build and manage an entire private swarm of agents? HivemindOS gives you shared memory, reusable skills, one-time setup, and a superbrain architecture for coordinating agent work.
| File | Size | Purpose |
|---|---|---|
swarm-sovereign-scout-Q4_K_M.gguf | ~6.9 GB | Recommended local/runtime artifact |
MTP/gemma-4-12B-it-MTP-Q8_0.gguf | ~444 MB | Draft model for llama.cpp MTP speculative decoding |
mmproj-swarm-sovereign-scout-bf16.gguf | ~167 MB | Projector artifact for llama.cpp multimodal image input |
chat_template.jinja | small | Swarm Scout identity/tool-use template override for llama.cpp |
The local workspace also produced a BF16 GGUF (swarm-sovereign-scout-bf16.gguf, ~22 GB). Q4_K_M is the recommended release/runtime target for speed and practicality.
The upstream base reports a large relative gain on local tau2-bench telecom-style agentic tool-use versus google/gemma-4-12B-it; see the upstream model card for the exact methodology and caveats.
Local verified artifact:
200/60.0086798667907714840/6 strict full-response refusals0/6 strict full-response refusalsThe strict check measures opening/full-response refusal behavior on a small local prompt set. It is not a broad safety benchmark and should not be treated as a guarantee of behavior in production. Add application-level guardrails where needed.
This release includes the Gemma 4 MTP draft model used for DSpark-style speculative decoding in llama.cpp.
Verified local build:
b9553, commit 9e3b928fd8c9d14dbf15a8768b9fdd7e5c721d66draft-mtp--spec-draft-n-min 0 --spec-draft-n-max 2MTP/gemma-4-12B-it-MTP-Q8_0.ggufVerified Modal build:
ghcr.io/ggml-org/llama.cpp:server-cuda-b9870-md MTP/gemma-4-12B-it-MTP-Q8_0.gguf --spec-type draft-mtpdraft_n / draft_n_accepted, confirming draft tokens were active.Older llama.cpp builds may not understand the Gemma 4 assistant draft architecture. For example, server-cuda-b9544 sees CUDA but fails to load this draft with unknown model architecture: 'gemma4-assistant'.
Clean local Apple Silicon measurements with reasoning disabled:
| Prompt set / config | Baseline tok/s | MTP tok/s | Speedup | Acceptance |
|---|---|---|---|---|
Mixed, n=0..2 | 48.12 | 70.41 | 1.46x | 0.75163 |
Code, n=0..2 | 33.96 | 47.93 | 1.41x | 0.69159 |
Mixed, n=1..3, p_min=0.05 | 37.54 | 60.14 | 1.60x | 0.63131 |
Use absolute tokens/sec as the tuning objective. The 1.60x row exists, but n=0..2 produced better real throughput on the local machine.
Download the model and draft:
hf download LiamVisionary/swarm-sovereign-scout-12b-GGUF \
swarm-sovereign-scout-Q4_K_M.gguf \
MTP/gemma-4-12B-it-MTP-Q8_0.gguf \
mmproj-swarm-sovereign-scout-bf16.gguf \
chat_template.jinja \
--local-dir swarm-scout-12b
Run an OpenAI-compatible server:
llama-server \
-m swarm-scout-12b/swarm-sovereign-scout-Q4_K_M.gguf \
-md swarm-scout-12b/MTP/gemma-4-12B-it-MTP-Q8_0.gguf \
--mmproj swarm-scout-12b/mmproj-swarm-sovereign-scout-bf16.gguf \
--spec-type draft-mtp \
--spec-draft-n-min 0 \
--spec-draft-n-max 2 \
-ngl 99 \
-ngld 99 \
-fa on \
--jinja \
--chat-template-file swarm-scout-12b/chat_template.jinja \
--reasoning auto \
--reasoning-format deepseek \
--alias swarm-scout-12b \
--host 127.0.0.1 \
--port 8080
Then call:
curl http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "swarm-scout-12b",
"messages": [{"role": "user", "content": "Write a small Python LRU cache."}],
"max_tokens": 512,
"temperature": 0
}'
Image input uses OpenAI-compatible content parts:
{
"model": "swarm-scout-12b",
"messages": [
{
"role": "user",
"content": [
{"type": "text", "text": "Read this image and summarize it."},
{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
]
}
],
"max_tokens": 256,
"temperature": 0
}
For fast agent loops, run with reasoning off:
--reasoning off --reasoning-format none
For harder debugging/planning, use:
--reasoning auto --reasoning-format deepseek
Compatible clients may also pass thinking_budget_tokens and chat_template_kwargs.enable_thinking per request.
The underlying Gemma 4 unified configuration includes image/video/audio processor metadata. The GGUF text model requires the matching mmproj-swarm-sovereign-scout-bf16.gguf projector for multimodal serving in llama.cpp.
Verified on 2026-07-05:
b9553 loaded swarm-sovereign-scout-Q4_K_M.gguf plus mmproj-swarm-sovereign-scout-bf16.gguf.server-cuda-b9870 advertised capabilities: ["completion", "multimodal"].Merchant: SWARM CAFE. Total: $8.37. through the production OpenAI-compatible endpoint.Image input is verified. The projector metadata also exposes an audio encoder, but audio/video input was not separately quality-tested for this release.
b9553; the Modal verified path used server-cuda-b9870; newer/older builds may need their own validation.google/gemma-4-12B-ityuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUFReleased under Apache 2.0, following the upstream/base lineage.