gl-agent-1-27b-GGUF

gl-agent-1-27b-GGUF

GGUF builds of a 27B model fine-tuned for long horizon agentic tasks. The quant, the MTP sidecar and the vision projector are separate downloads, so you take only what you use.

Which file

Perplexity is measured on held-out agentic traces that the calibration data never saw. The absolute values mean nothing outside this table; they are here to show where the curve bends. BF16 reference is 2.6333.

Q8 down to Q4 costs 0.033 perplexity, so there is little reason to run anything above UD-Q4_K_XL. UD-IQ3_XXS is where the size saving stops being free.

UD-IQ1_S_XS is here because sub-2-bit is interesting, not because it works. It writes coherent prose on some prompts and drops into a repetition loop on others. Do not put an agent behind it.

Running it

llama-cli -hf Loke-60000/gl-agent-1-27b-GGUF:UD-Q4_K_XL \
  -ngl 99 -c 32768 --jinja

Needs a recent llama.cpp. This architecture mixes full attention with linear attention layers and support for it is new; build b1-5de25a7 or later works. Set -c to what your memory allows, since the model's own maximum is far larger than most single machines can hold.

Thinking is on by default. enable_thinking, preserve_thinking and reasoning_effort are settable per request through the chat template.

MTP sidecar, optional

The multi-token-prediction block ships as its own file. llama.cpp ignores those tensors entirely during ordinary decoding, so folding them into every quant would make everyone carry weight that most people never execute.

llama-cli -m gl-agent-1-27b-UD-Q4_K_XL.gguf \
  -md mtp-gl-agent-1-27b-q8_0.gguf \
  --spec-type draft-mtp -ngl 99 -c 32768 --jinja

Measured on UD-Q4_K_XL at a fixed seed: 11.4 t/s without it, 17.4 t/s with it. Take the sidecar nearest your quant. q8_0 is 3.16 GB, q4_K is 2.03 GB, and the BF16 copy is there if you want to quantise it yourself. The mtp- prefix is the sidecar convention, so -hf together with --spec-type draft-mtp resolves it without being told where to look.

The sidecar carries its own embedding and output tensors so it can load as a draft model, which means the pair comes to a little more than a fused build would. That is the price of not imposing it on everyone.

Vision, optional

llama-mtmd-cli -m gl-agent-1-27b-UD-Q4_K_XL.gguf \
  --mmproj mmproj-gl-agent-1-27b-F16.gguf -ngl 99

How these were built

The importance matrix was computed over real agentic trajectories rather than generic web text, so the tensors that carry tool loops are the ones weighted during quantisation.

Bit depth is not uniform. FFN tensors are 64% of the weights and that is where the bits come from. Token embeddings, the output head and the full-attention layers sit one to several steps higher, and the small state tensors of the linear-attention layers stay at full precision. Those are what break first and they are cheap to keep.

Every file above was checked for a valid header, scored on the held-out set, and made to generate before being published.

Benchmarks

Coming soon.