Alpamayo2-Super-mlx-4bit

Alpamayo 2 Super · MLX 4-bit

A 34B driving model on a laptop. It answers in about 14 seconds, which is why this is a curiosity and not a driving stack.

What this is

The vision-language model of nvidia/Alpamayo2-Super converted to MLX and quantized to 4 bits, paired with the original diffusion expert. It runs on Apple-silicon Macs.

vlm/       the VLM, in MLX format — 4-bit affine, group size 64, 23 GB on disk
expert/    the diffusion expert, unchanged from upstream (PyTorch)
mlx/       the conversion tool and the bridge that joins the two

torch alone cannot hold Alpamayo 2 Super on a 64 GB Mac. The bridge runs the VLM under MLX and hands its KV cache to the PyTorch expert, so only the expert needs MPS memory.

Usage

This is not a drop-in from_pretrained model. It needs the bridge in mlx/mlx_bridge.py together with the patched upstream code: drivetechodyssey-tech/alpamayo1.5, branch apple-silicon.

from mlx_bridge import MlxBridge

model = MlxBridge("drivetechodyssey/Alpamayo2-Super-mlx-4bit")          # or a local path to this folder
out = model.generate(inputs)         # trajectory samples + reasoning text

Speed and memory

Measured on an M5 Max (64 GB).

Quality: one run, and no baseline is possible here

One official clip, 5,328 tokens, 7 cameras, 1 sample: 14.22 s (VLM 12.56 + expert 1.66), 34.0 GB peak, minADE 2.506 m. Reasoning text: "Nudge to the left to pass the construction vehicle encroaching into the lane."

There is no bf16 comparison for this model and there cannot be one on a 64 GB Mac — building it in torch at bf16 needs about 72 GB, which is the reason this conversion exists. So the effect of 4-bit quantization on Alpamayo 2 Super is unmeasured.

For the smaller sibling, 4-bit did not degrade the reasoning text on the clip it was tested on (see the Alpamayo 1.5 repos). That is a hint, not evidence about this model.

Use this for experiments, not for anything that decides what a vehicle does.

Notes

• The expert is unchanged. Only the VLM was converted.
• mx.random.seed() in MLX returns a first value that ignores the sampling distribution. The bridge discards it; without that, the first token of every plan is effectively a coin flip.
• Requires the Apple-silicon patches to the upstream inference code (MPS autocast, SDPA, device-name handling). They live in the fork above; upstream is NVlabs/alpamayo1.5.

Origin and license

Derived from nvidia/Alpamayo2-Super by NVIDIA, licensed under OpenMDW-1.1. The upstream licence is included as LICENSE / ORIGINAL_LICENSE, and the upstream model card as ORIGINAL_README.md, as that licence requires.

The conversion adds no new training data and no new weights — it is a numeric format change of the published parameters.