Qwythos-27B-v1-MLX-VLM-bf16

Qwythos-27B-v1 · MLX VLM BF16

Unquantized bfloat16 MLX conversion of empero-ai/Qwythos-27B-v1 for Apple Silicon. Vision tower kept.

Not the everyday download. Prefer 4-bit VLM on ≤32–48GB Macs. This build is ~55GB and needs high unified memory.

Which repo should I download?

Install

pip install -U mlx-vlm

Quick start

mlx_vlm.generate \
  --model FancieF/Qwythos-27B-v1-MLX-VLM-bf16 \
  --prompt "Introduce yourself in one sentence." \
  --thinking-mode disabled \
  --max-tokens 256 \
  --temperature 0.6

Vision

mlx_vlm.generate \
  --model FancieF/Qwythos-27B-v1-MLX-VLM-bf16 \
  --image ./photo.jpg \
  --prompt "Describe this image." \
  --thinking-mode disabled

+ MTP

mlx_vlm.generate \
  --model FancieF/Qwythos-27B-v1-MLX-VLM-bf16 \
  --draft-model FancieF/Qwythos-27B-v1-MLX-MTP \
  --draft-kind mtp \
  --draft-block-size 4 \
  --prompt "..." \
  --thinking-mode disabled

LM Studio / memory

• BF16 27B + KV easily exceeds 32GB — expect swap or OOM on common MacBooks.
• For LM Studio with MTP, use official GGUF MTP quants instead of this MLX BF16.

What’s inside

• MLX safetensors, BF16, no weight quantization
• language_model + vision_tower
• No baked-in MTP (same as 4-bit; pair with the MTP repo)

Conversion

mlx_vlm.convert \
  --hf-path /path/to/Qwythos-27B-v1 \
  --mlx-path ./Qwythos-27B-MLX-VLM-bf16 \
  --dtype bfloat16 --trust-remote-code

(No -q.)

License & attribution

Derived from empero-ai/Qwythos-27B-v1 (Apache-2.0), based on Qwen3.5-27B. Unofficial MLX conversion.

Links