Agents-A1-4B-mxfp4-mlx

Agents-A1-4B — Unsloth MXFP4 + MXFP8 (MLX)

Brooooooklyn/Agents-A1-4B-mxfp4-mlx is an Apple Silicon MLX quantization of InternScience/Agents-A1-4B. Agents-A1-4B is the 4B dense member of the Agents-A1 agentic family: a Qwen3.5-architecture vision-language model with 32 language-model layers in a hybrid attention stack — 8 full-attention layers at indices 3, 7, 11, 15, 19, 23, 27, 31 and 24 gated-delta linear-attention layers — plus a 24-block vision tower. It has 4,539,265,536 parameters, ties its input and output embeddings, and declares mtp_num_hidden_layers: 1 although the published checkpoint ships no mtp.* tensor subtree.

This model is part of the Qwen Unsloth tensor-class recipe for MLX on macOS and DGX collection.

The source was the BF16 checkpoint at revision 945c40a4aa6f534d434a353207b8d42ecf7a5293.

Update — re-converted 2026-09-01

The MXFP4 and MXFP8 weights are re-encoded. mlx-node now picks each MX block's shared E8M0 exponent itself instead of rounding log2(amax) to nearest: MXFP4 tries both candidate exponents and keeps the lower squared error, and MXFP8 takes the ceiling so a block can never saturate. Measured MXFP8 weight error across a checkpoint drops from 6.91% to 2.66%.

Weight bytes changed; the tensor-class map, shard layout and tensor counts are unchanged. Earlier revisions remain in this repository's history.

Quantization recipe

This is a data-free, weight-only Apple translation of the Unsloth Qwen3.6 NVFP4 recipe:

• tensors assigned NVFP4 by the recipe are stored as MXFP4, 4-bit with group size 32;
• tensors assigned FP8 by the recipe are stored as MXFP8, 8-bit with group size 32;
• every excluded tensor stays BF16.

This ports the recipe's tensor-class selection. It does not claim numerical parity with NVIDIA NVFP4/FP8 activation quantization. Both MXFP classes use MLX weight-only quantized matmul with BF16/A16 activations. The artifact contains no activation-calibration or FP8 KV-cache-calibration data, global scales, or W4A4/W8A8 execution metadata. No imatrix, calibration dataset, or AWQ-style pre-scaling was used, and no upstream numerical or performance parity is claimed.

That is 72 MXFP4 modules and 128 MXFP8 modules, 200 in total. The final eight FFN layers are intentionally MXFP8; this is the regular, accuracy-oriented recipe rather than an all-FFN-FP4 "Fast" variant.

lm_head differs from the 35B sibling

Agents-A1-4B sets tie_word_embeddings: true. The source checkpoint therefore contains no lm_head.weight, and the converter drops the tied head rather than materializing it. The output head stays BF16, sharing the BF16 embed_tokens matrix. This is the one place where this artifact departs from Agents-A1-mxfp4-mlx, where lm_head is an untied MXFP8 tensor.

A note on the top-level quantization block

All 200 quantized modules carry an explicit mxfp4 or mxfp8 override, so the top-level default resolves nothing. Read the per-module entries, not the top-level block.

For fixed-map recipes mlx-node 0.0.8 emits {"bits": 3, "group_size": 64, "mode": "affine"} as that default. It is inert at load time but misleading as metadata — the Hub's quantization auto-tagger reads it and labels the repo 3-bit. In this repository the default is rewritten to {"bits": 4, "group_size": 32, "mode": "mxfp4"} in both the quantization and quantization_config mirrors, which must stay identical or mlx-node refuses to load. A load-and-generate A/B at temperature: 0 produced byte-identical output before and after the rewrite.

The NVFP4 sibling needs no such fixup: its top-level default is a real nvfp4 4/16 that the 72 low-class modules genuinely inherit.

Compatibility and usage

This checkpoint requires @mlx-node/lm and @mlx-node/core 0.0.8 or newer. The published prebuilt macOS binary requires macOS 26 or newer; macOS 14 or newer can use a source build.

npm install @mlx-node/lm@^0.0.8 @mlx-node/core@^0.0.8

import { loadSession } from '@mlx-node/lm';

const session = await loadSession('./Agents-A1-4B-mxfp4-mlx');
const result = await session.send('Reply briefly: what can you help with?');
console.log(result.text);

Reproduction

Converter: mlx-node at or after PR #131, which made the tuned MX weight encoders and the NVFP4 power-of-two lift unconditional. v0.0.8 reproduces the earlier revision of this repository, not the current weights. The reproducible invocation from the mlx-node repository root was:

mlx convert \
  --input .cache/models/agents-a1-4b \
  --output .cache/models/agents-a1-4b-unsloth-mxfp4-mlx \
  --model-type qwen3_5 \
  --quantize \
  --q-recipe unsloth \
  --q-mxfp

No --imatrix-path was supplied. For the fixed MXFP map an imatrix is optional and only controls AWQ pre-scaling; the tensor-class assignment is identical either way. Quality may be lower than a calibrated conversion.

Validation

A local mlx-node load-and-generate smoke test loaded the checkpoint in 608 ms and produced coherent, on-topic English at temperature: 0. This functional smoke test does not validate model quality.

Benchmark

Measured on an Apple M5 Max with 128 GiB of unified memory, Darwin 25.5.0/arm64, Node.js 24.13.1, and @mlx-node/lm, @mlx-node/core, and @mlx-node/core-darwin-arm64 0.0.8.

The table reports the median of 3 measured runs. Each run used a fresh child process and fresh model load, with 0 warmups and a 60-second cooldown. Generation was deterministic: temperature: 0, reasoning effort none, 106 prompt tokens, and a 512-token generation limit. All 3 runs generated 512 tokens and ended with finishReason: "length".

Raw per-run samples and the machine-readable summary are available in benchmark.json.

Across the three runs, decode ranged from 55.5 to 87.4 tok/s and model load from 0.533 to 0.687 s. Load timing is sensitive to storage and OS file-cache state, and sustained Apple Silicon throughput is sensitive to thermal state. Treat these figures as host- and cache-specific. They describe only this prompt and configuration and must not be used to infer model quality.

License and attribution

The source model card declares the Apache-2.0 license. Model capability and training credit belong to InternScience. The tensor-class recipe is credited to Unsloth, and this repository only converts the BF16 source weights to an MLX-native mixed MXFP4/MXFP8 representation.