Hy-MT1.5-1.8B-2bit-onnx

Hy-MT1.5-1.8B (ONNX)

ONNX exports of Tencent's Hy-MT1.5-1.8B translation model in two precisions, produced with mobius.

Naming note: the upstream tencent/Hy-MT1.5-1.8B-2bit repo ships BF16 safetensors; the true 2-bit (SEQ) weights live in AngelSlim/Hy-MT1.5-1.8B-2bit-GGUF. Both are mirrored here as ONNX.

Both contain model.onnx + model.onnx.data and the original tokenizer / chat template.

Build commands

# bf16/
python -m mobius build --model tencent/Hy-MT1.5-1.8B-2bit \
    --dtype bf16 --optimize --runtime ort-genai ./bf16

# Q1_0/
huggingface-cli download AngelSlim/Hy-MT1.5-1.8B-2bit-GGUF \
    Hy-MT1.5-1.8B-2bit.gguf --local-dir ./gguf
python -m mobius build-gguf ./gguf/Hy-MT1.5-1.8B-2bit.gguf \
    -o ./Q1_0 --keep-quantized --dtype f32

Q1_0 details

The Q1_0/ folder uses Tencent's Stretched Elastic Quantization (SEQ) codebook {−3, −1, +1, +3} · stored_scale, 512-element blocks. mobius packs this into ORT MatMulNBits two ways:

• Default (used here): bits=4 inflated form — each 2-bit code c becomes 2c ∈ {0,2,4,6}, integer zero_point=3. Exercises ORT's well-optimised packed-uint8 path; ~33 tok/s on CPU EP. 2× the source weight bytes but fast.
• Opt-in via MOBIUS_TENCENT_Q1_0_USE_NATIVE_2BIT=1: bits=2 native — codes pass through, float zero_point=1.5. Matches the source 2 bpw. Requires onnxruntime ≥ 1.27 (microsoft/onnxruntime#28354); CPU throughput is currently ~0.24 tok/s pending an MLAS fast path (microsoft/onnxruntime#28552).

Per-tensor dequantization matches HF safetensors to bf16-rounding precision (max abs diff 2.3e-5). Short-prompt greedy generation produces token-for-token identical output to HF PyTorch.

License

Inherits the Tencent HunYuan Community License from the upstream model.