dam-vision-v3-mlx

dam-vision-v3-mlx (8-bit)

MLX 8-bit build of the DAM vision model — a fine-tuned Qwen2-VL-2B-Instruct with the DAM v3 LoRA merged into the base weights. This is the same fine-tuned model the DAM pipeline serves via vLLM (tag qwen2-vl-2b-v3+f2), packaged here to run natively on Apple Silicon Macs with MLX (Metal) — no server, no CUDA.

Details

Usage

pip install mlx-vlm
python -m mlx_vlm generate \
  --model Munchit/dam-vision-v3-mlx \
  --image your_frame.jpg \
  --prompt "Describe this image." \
  --max-tokens 128 --temperature 0.0

Python:

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("Munchit/dam-vision-v3-mlx")
config = model.config
prompt = apply_chat_template(processor, config, "Describe this image.", num_images=1)
out = generate(model, processor, prompt, ["your_frame.jpg"], max_tokens=128, temp=0.0)
print(out)

Known limitation (v3)

The v3 fine-tune has a known caption-repetition failure mode: a meaningful fraction of outputs loop/repeat catastrophically. This is inherent to the v3 weights — it is not introduced by the MLX conversion and is not fixable by decode settings or prompt changes; it requires a v4 retrain. Keep this in mind before relying on it for production captioning.

Provenance

Converted on Linux (MLX CPU build) from the merged v3 checkpoint; the MLX weight format is portable and runs on Apple Silicon under Metal. Structural load + the underlying weights are validated (the same model runs in the DAM vLLM pipeline); verify end-to-end generation on an Apple Silicon device.