Alpamayo-1.5-10B-mlx-4bit

Alpamayo 1.5 10B · MLX 4-bit

The smallest one. Read the quality note before using it.

What this is

The vision-language model of nvidia/Alpamayo-1.5-10B converted to MLX and quantized to 4 bits, paired with the original diffusion expert. It runs on Apple-silicon Macs.

vlm/       the VLM, in MLX format — 4-bit affine, group size 64, 9.9 GB on disk
expert/    the diffusion expert, unchanged from upstream (PyTorch)
mlx/       the conversion tool and the bridge that joins the two

torch alone cannot hold Alpamayo 1.5 on a 64 GB Mac. The bridge runs the VLM under MLX and hands its KV cache to the PyTorch expert, so only the expert needs MPS memory.

Usage

This is not a drop-in from_pretrained model. It needs the bridge in mlx/mlx_bridge.py together with the patched upstream code: drivetechodyssey-tech/alpamayo1.5, branch apple-silicon.

from mlx_bridge import MlxBridge

model = MlxBridge("drivetechodyssey/Alpamayo-1.5-10B-mlx-4bit")          # or a local path to this folder
out = model.generate(inputs)         # trajectory samples + reasoning text

Speed and memory

Measured on an M5 Max (64 GB).

Quality: measured once, not ranked

One official clip, 3,086 tokens, 4 cameras, 6 trajectory samples, diffusion noise pinned on both sides so the comparison is like for like:

All three produced the same reasoning text.

Do not read the minADE column as a ranking. With one clip and one seed the spread is dominated by which of the six samples happened to land closest to the ground truth. The defensible claim is that 4-bit did not visibly degrade on this clip — not that it beats bf16. Ranking these would take several clips across several seeds, which has not been done.

8-bit was not in that comparison.

Precision changes decoding, not prefill. Prefill runs at 2,160 / 1,993 / 2,046 tok/s for bf16 / 8-bit / 4-bit — the same within noise. Generation is 32 / 59 / 98 tok/s. The bridge's advantage over torch is the framework, not the quantization: torch on MPS prefills at 323 tok/s, about a sixth of MLX.

Notes

• The expert is unchanged. Only the VLM was converted.
• mx.random.seed() in MLX returns a first value that ignores the sampling distribution. The bridge discards it; without that, the first token of every plan is effectively a coin flip.
• Requires the Apple-silicon patches to the upstream inference code (MPS autocast, SDPA, device-name handling). They live in the fork above; upstream is NVlabs/alpamayo1.5.

Origin and license

Derived from nvidia/Alpamayo-1.5-10B by NVIDIA, licensed under OpenMDW-1.1. The upstream licence is included as LICENSE / ORIGINAL_LICENSE, and the upstream model card as ORIGINAL_README.md, as that licence requires.

The conversion adds no new training data and no new weights — it is a numeric format change of the published parameters.