Agnes-3.0-Flash-GGUF

Agnes 3.0 Flash GGUF

[!IMPORTANT] These GGUFs do not load on stock llama.cpp yet. Agnes runs a second, narrower SwiGLU in parallel with the main FFN on every layer, and upstream has no support for it, so a stock build stops with:

error loading model: done_getting_tensors: wrong number of tensors; expected 1188, got 972

(the 216 missing tensors are the parallel branch; the 15 MTP tensors are dropped too). Until upstream lands support, use our fork:

git clone -b agnes-3.0-flash https://github.com/JakeATX/llamAmpere.git
cd llamAmpere && cmake -B build -DGGML_CUDA=ON && cmake --build build -j --target llama-server

Details: docs/agnes-3.0-flash.md

Quantization build for the Preview release of Agnes-AI/Agnes-3.0-Flash, pinned to source revision 24f712ce59379b54c4a141d2708c35daf5ff613b.

Build status

The source safetensors are downloaded and verified locally. Conversion and uploads are in progress; quant folders will be published independently as they finish:

• ATX-Q3_K_XL/
• ATX-Q4_K_XL/
• ATX-Q4_K_M/
• ATX-Q5_K_XL/
• ATX-Q6_K_XL/
• ATX-Q8_K_XL/
• ATX-IQ4_XS-M/

Every target GGUF includes Agnes' available one-layer MTP/NextN head. The GGUFs also retain the checkpoint's second 2,048-wide parallel FFN branch.

All Agnes model conversion, model-specific recipe adaptation, quantization, validation, and publication in this repository were performed by ATX. Unsloth did not produce these Agnes quants. Where an Unsloth Qwen3.8 calibration asset is named in GGUF metadata, that records an input to ATX's layer-role transfer; it is not an authorship or publisher claim for these artifacts.

Compatibility

Agnes' parallel FFN is not supported by stock llama.cpp at publication time. The artifacts are built from official ggml-org/llama.cpp commit 3057bb66c86c46d5781e50e85462a760ba7d1feb plus the narrowly scoped patch in build-info/. The patch commit 5bb7de853836418731d655a287af645bdf8257a5 has that official commit as its direct parent. These artifacts contain the text model and native MTP head. The vision projector is not included because the current converter does not support an Agnes mmproj export.

The model is dense rather than MoE. The dynamic quant recipes therefore apply role-aware bit allocation across recurrent attention, global attention, main FFN, and parallel FFN tensors instead of routed-expert formulas. The parallel FFN follows the corresponding main FFN tensor type in each layer.