Hy3-oQ2e-2.37bpw

Hy3 (MLX, oQ2e @ 2.37 bpw)

Calibrated 2-bit MLX quantization of tencent/Hy3 (Hunyuan 3.0, 295B-A21B MoE), produced with omlx oQe at level 2. Effective 2.37 bits/weight over quantizable weights, 87.7 GB on disk. Targets Apple Silicon.

This is a shell-reduced variant of mlx-community/Hy3-oQ2e: the routed experts are identical, but the non-expert layers (attention, embeddings, lm_head) are quantized below the 8-bit that oQ2e keeps, trading a small amount of quality for ~2 GB.

Quantization layout

Routed experts are bit-for-bit the same as oQ2e (imatrix reused from the same calibration cache). Only the shell changed.

How it was quantized

Built from the BF16 source (~550 GB / 591 GiB) with omlx oQ level 2 + imatrix weighting. To stay within 128 GB RAM: the sensitivity pass reused the existing oQ2 quantization instead of building a full-precision proxy, and the importance matrix was reused from the oQ2e calibration cache. Streamed tensor-by-tensor.

Requirements

Runs natively in oMLX. For mlx-lm, hy_v3 support is pending upstream; until then:

uv pip install "mlx-lm @ git+https://github.com/kernelpool/mlx-lm.git@add-hy3-preview"

Benchmarks (all Hy3 MLX variants)

oMLX intelligence suite, 300 seeded samples per benchmark, identical questions across models. I ran seeded samples — this is not a complete benchmark run, so read the differences as noise and test the versions against your own workload before picking one.

Variants: oQ2 · oQ2e · oQ2e-2.37bpw · oQ2e-2.33bpw · oQ2e-2.31bpw

Usage

python -m mlx_lm generate --model mlx-community/Hy3-oQ2e-2.37bpw \
  --prompt "Explain Bayes' theorem in two sentences." --max-tokens 300

from mlx_lm import load, generate
model, tokenizer = load("mlx-community/Hy3-oQ2e-2.37bpw")

License

Apache-2.0, inherited from tencent/Hy3.