Downloads · 30 days
75
10% of all-time downloads
alexop1000/Agents-A1-Q8_0-GGUF
Agents-A1-Q8_0-GGUF is a image-text-to-text model from alexop1000. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for llama.cpp. The card lists the license as apache-2.0.
GGUF quantization of InternScience/Agents-A1 for llama.cpp, including the vision projector so the model keeps its multimodal (image + text) capability.
Downloads · 30 days
75
10% of all-time downloads
All-time downloads
774
Public
Repo size
74.7 GB
Likes
1
Public
Click a slice to open those files.
.gguf37.8 GB · 100%
From the Hugging Face model README
GGUF quantization of InternScience/Agents-A1 for llama.cpp, including the vision projector so the model keeps its multimodal (image + text) capability.
| File | Type | Size | Notes |
|---|---|---|---|
agents-a1-q8_0.gguf | text model, Q8_0 | 36.9 GB | chat template + tokenizer embedded |
mmproj-Agents-A1-f16.gguf | vision projector, F16 | 0.9 GB | required for image input |
chat_template.jinja | chat template | — | already baked into the GGUF; loose copy for --chat-template-file overrides |
Converted with llama.cpp/convert_hf_to_gguf.py (architecture qwen35moe, registered upstream as Qwen3_5MoeForConditionalGeneration) using --no-mtp.
The upstream config.json declares mtp_num_hidden_layers: 1, but the published checkpoint contains no MTP / nextn weights (no layers.40.*, no eh_proj/enorm/hnorm/shared_head; highest layer index is 39). Any converter that honors the config bumps block_count to 41 and writes nextn_predict_layers, yet cannot fill block 40 — yielding a file that declares 41 blocks but ships 40, which fails to load with missing tensor 'blk.40.attn_norm.weight'. --no-mtp keeps block_count=40 to match the 40 real layers that actually shipped. MTP is only a speculative-decoding accelerator, so nothing is lost for normal inference.
The chat template embedded in the GGUF is byte-identical to the upstream chat_template.jinja and includes the vision-token, tool-call, and <think> reasoning logic.
[!IMPORTANT] Agents-A1 uses a hybrid linear-attention + MoE architecture (Qwen3.5-MoE family). You need a recent llama.cpp build that supports the
qwen35moeinference graph — converter support landing does not guarantee your local binary can run it. If the model fails to load, update and rebuild llama.cpp frommaster.
Download both GGUFs, then point --mmproj at the projector:
llama-server -m agents-a1-q8_0.gguf --mmproj mmproj-Agents-A1-f16.gguf -c 8192 -ngl 99
Then send images via the web UI or the OpenAI-compatible image_url message format.
llama-cli -m agents-a1-q8_0.gguf -p "Hello"
Omitting --mmproj runs the language model alone and frees the compute/KV the vision encoder would use.
Agents-A1 is a 35B Mixture-of-Experts agentic model from InternScience, built to scale heterogeneous agentic abilities across long-horizon search, engineering, scientific research, instruction following, and tool-calling. Despite the ~35B class, it reports competitive-to-SOTA results against much larger frontier systems on benchmarks such as Seal-0, HiPhO, FrontierScience, IFBench, BrowseComp, and GAIA.
See the original model card for full details, benchmarks, and licensing. All credit for the model goes to InternScience; this repo only provides GGUF conversions. License: Apache-2.0.