agent-a1-alchemist-gguf

Agent-A1 (The Alchemist) — GGUF

The same weights as angelgalvisc/agent-a1-alchemist-4bit, repacked for llama.cpp so the model runs where MLX does not: Android, iOS, NVIDIA, plain CPU.

2.92 GB, and 0.67 GB more if you want it to see.

It is the same model, not a new quantization

The 4-bit codes were unpacked from the MLX artifact and written straight into Q4_1 blocks. Nothing was rounded a second time:

• 248 projections at Q4_1. Q4_1 reconstructs w = q·d + m with d and m in fp16, which is the arithmetic the original uses, so a group of 128 becomes four blocks of 32 sharing one scale and one minimum. Codes, scales and minima are identical bit for bit — checked tensor by tensor before writing. Every scale crosses bf16 → fp16 untouched; the smallest in this model is 7.9e-5 and fp16 reaches down to 6.1e-5.
• Vocabulary table at Q8_0. It is stored at 6 bits and GGUF has no affine 6-bit type — the affine types stop at 5 bits and the 6-bit one is symmetric — so this is the one tensor that was rounded again. It was measured first: across 194 positions of real context the most likely next token never changed, and the top five agreed 99.7% of the time. Keeping it exact instead costs 600 MB.

Asked the same six questions with greedy decoding, this file and the MLX artifact answer identically word for word on four of six, and reach the same result on the other two by a different route. Where they diverge, llama.cpp is the more precise of the two: it reconstructs the weights in fp32 while MLX does it in bf16.

Run it

llama-cli -m agent-a1-alchemist-4B-Q4_1.gguf -p "How much is 17 x 23?"

Or from Python:

from llama_cpp import Llama
llm = Llama(model_path="agent-a1-alchemist-4B-Q4_1.gguf", n_ctx=4096, n_gpu_layers=-1)
print(llm.create_completion("...", max_tokens=300, temperature=0.0)["choices"][0]["text"])

Thinking is a switch, and it is on by default here

The chat template writes a reasoning block inside <think>...</think> unless it is told not to. Every figure this model was measured on came from thinking off. To turn it off, end the prompt with an empty block:

<|im_start|>assistant
<think>

</think>

With thinking on, leave room: a substantive answer runs about 2800 tokens against 1600 direct.

Images and video

Add the projector and it sees:

llama-mtmd-cli -m agent-a1-alchemist-4B-Q4_1.gguf \
               --mmproj mmproj-agent-a1-alchemist.gguf \
               --image plan.png -p "What is this?"

The image encoder was never compressed — these are the original weights at fp16, which is why the projector weighs what it does. Handed a scanned engineering drawing it answers "a wastewater treatment plant", the same as the MLX build. It reads what a picture is well and misreads the digits printed on it, so use it to know what you are looking at, not to take figures off it.

One thing to know

No multi-token prediction. The upstream config declares one MTP block, but neither this model nor InternScience's own release ships its weights — 723 tensors on both sides, none of them nextn. Converting this architecture with current llama.cpp leaves nextn_predict_layers = 1 and the runtime then asks for a block nobody has; here it is set to 0. Anyone converting Agents-A1 themselves will meet the same thing.

What was done to the model

Nothing was retrained. 216 of the 248 matrices are rescaled per channel before rounding, with the compensating division folded into the neighbouring layer, so the model computes the same thing at the same file size. The other 32 are the attention outputs, which have no earlier layer to absorb it. No range is clipped.

On a 60-task bench where the model plans and calls real tools, this scores 48 against the 43 of the previously released 4-bit version.

License

Apache 2.0 for this packaging. The model is InternScience's, under its own terms. See NOTICE in the MLX repository for the attribution chain.