Agents-A1-bf16

Agents-A1 — MLX (bf16)

MLX conversion of InternScience/Agents-A1, in bf16. The source checkpoint is already bf16, so this is a lossless format conversion — not a quantization.

Agents-A1 is a Qwen3.5-MoE vision-language agent model (qwen3_5_moe, Qwen3_5MoeForConditionalGeneration): 40 decoder layers, 256 routed experts per layer + a shared expert, hidden size 2048, with a vision tower and video preprocessing.

Running it

Multimodal (VLM) — load with mlx-vlm (mlx-lm can't load multimodal architectures):

pip install mlx-vlm
python -m mlx_vlm.generate --model mlx-community/Agents-A1-bf16 \
  --prompt "What is 17 * 24? Think step by step." --max-tokens 512
# with an image:
python -m mlx_vlm.generate --model mlx-community/Agents-A1-bf16 --image img.jpg --prompt "Describe this image."

Loads and runs in stock mlx-vlm — no patched code needed at inference.

Throughput

Measured with oMLX's benchmark harness on a Macbook Pro M5 Max 128GB 40 GPU — gen 128 tokens, cold prefill (unique prompt prefix per request, no cache reuse).

Single request (batch 1) — decode tok/s by context

TTFT (cold prefill) is ~precision-independent — ≈0.3 s @1k, 3 s @8k, 21 s @32k, 63 s @64k, ~225 s @128k — prefill is compute-bound, not weight-bound.

Continuous batching (1k context) — aggregate decode tok/s

Aggregate across the batch; per-request rate is that value divided by the batch size.

Smoke test

17 x 24 -> correct (408), coherent, no repetition.

Other precisions

License

apache-2.0, inherited from the base model.