HY-Embodied-0.5-bf16-mlx

HY-Embodied-0.5 — bf16 MLX

BF16 MLX conversion of tencent/HY-Embodied-0.5, a 3.79B mixture-of-transformers embodied VLM, running natively on Apple Silicon. This is the reference every measurement in the quantization ladder was made against: it matches the pinned upstream implementation token-for-token over the committed golden fixtures, in both thinking modes. These weights require the from-scratch MLX implementation of the hunyuan_vl_mot architecture published alongside them: hy-embodied-mlx. No other public runtime supports this architecture.

What was modified

BF16 safetensors converted to MLX layout; auto_map removed; nothing quantized.

Measured

Text-only probe (50 scripted prompts): 0.700 [0.562, 0.809] no-think, 0.900 [0.786, 0.957] think.

Brackets are Wilson 95% intervals (n=100 pointing, n=50 probe). Measured on an M3 Max (36 GB), greedy decoding; per-question CSVs, statistical addendum, and one-command reproduction live in the implementation repo. Quantized variants measured against this reference: 8-bit, 6-bit, 5-bit, 4-bit.

All variants and the runtime are collected at https://huggingface.co/collections/vimalnakrani/hy-embodied-05-mlx-6a550eb39f59d2adf90c0355.

The "Use this model" snippet Hugging Face auto-generates for MLX repos (mlx-vlm) does not support this architecture; the Usage section below is the working path.

Usage

from PIL import Image
from transformers import AutoTokenizer
from hy_embodied_mlx.model import load, generate
from hy_embodied_mlx.pointing import FORMAT_INSTRUCTION
from hy_embodied_mlx.processor import Processor

model_dir = "HY-Embodied-0.5-bf16-mlx"
tok = AutoTokenizer.from_pretrained(model_dir)
model = load(model_dir)
messages = [{"role": "user", "content": [
    {"type": "image"},
    {"type": "text", "text": f"Point to the red mug in the image. {FORMAT_INSTRUCTION}"},
]}]
inputs = Processor(tok)(messages, images=[Image.open("desk.jpg")])
print(tok.decode(generate(model, inputs, max_tokens=128)))

Pointing needs the format instruction shown — a bare "point to X" gets a prose location description. Emitted coordinates are integers in 0-1000, normalized to the preprocessed canvas (Tencent's documentation does not specify the frame; for images whose dimensions are multiples of 32 and within the 2048x2048 pixel budget, the canvas is pixel-identical to the input image). Thinking mode is controlled with enable_thinking=True/False on the chat template.

License

These weights are a Model Derivative of Tencent HY, distributed under the Tencent HY Community License (full text in the LICENSE file; NOTICE included). This is not an open-source license. The obligations and restrictions pass through to you:

• Territory: the license does not grant rights in the European Union, the United Kingdom, or South Korea.
• The Section 5(a) acceptable-use restrictions and the Section 5(b) restriction — including not using this model or its outputs to improve any other AI model — apply to these weights and anything you build on them.
• If you redistribute these weights or derivatives of them, include a copy of the license agreement, the NOTICE file, and a prominent statement of what you modified.

These Model Derivatives are distributed by the Hugging Face account vimalnakrani. This repository is an independent conversion and is not affiliated with, sponsored, or endorsed by Tencent.