MiLMMT-46-1B-v0.1-4bit

MiLMMT-46-1B-v0.1 — MLX 4-bit

Apple MLX weights for xiaomi-research/MiLMMT-46-1B-v0.1, Xiaomi's 46-language translation model (a translation fine-tune of Gemma 3 1B).

Read this before using it

Use penta2himajin/MiLMMT-46-1B-v0.1-6bit instead.

Naive 4-bit measurably damages this model, and every regression it introduces disappears at 6 bits — no calibration, just --q-bits 6. The 6-bit build is 269 MB larger and is the one you want. This repository stays up so the comparison remains reproducible, not because it is a good artifact.

The Q4_K_M GGUF, penta2himajin/MiLMMT-46-1B-v0.1-Q4_K_M-GGUF, is also a better choice than this build, and loads ~4× faster than either MLX build because llama.cpp mmaps its weights.

Both were run on the same 16-sentence JA↔EN set, on the same machine, with a byte-identical prompt (the format below) and greedy decoding, so the difference is attributable to the quantization rather than to the harness. Timings are from a battery-powered run and are useful as a ratio, not as absolute figures.

Regressions this conversion introduces, none of which the GGUF produces:

Small models are disproportionately sensitive to naive post-training quantization. Raising the bit width to 6 closes the gap completely — every row in the table above is fixed in the 6-bit build, which lands at 1.0 GB and 204 chars/s. Calibrated quantization (mlx_lm.dwq, mlx_lm.awq) might reach that quality at this size, but was not needed and is untried.

Prompt format

MiLMMT ships no chat template. Do not apply one — prompt it raw, exactly as the upstream model card specifies:

Translate this from <source language name> to <target language name>:
<source language name>: <source sentence>
<target language name>:

Use the language names from the upstream card's supported-language list (e.g. Japanese, English). Recommended decoding: top_k=1, temperature=0.

pip install mlx-lm
PROMPT=$'Translate this from Japanese to English:\nJapanese: こんにちは、今日も一日頑張りましょう。\nEnglish:'
mlx_lm.generate --model penta2himajin/MiLMMT-46-1B-v0.1-4bit \
  --prompt "$PROMPT" --max-tokens 128 --temp 0.0

Because there is no turn marker, generation can run on into a second translation block. Stop on the string Translate this from, or trim at it.

Conversion

mlx_lm.convert --hf-path xiaomi-research/MiLMMT-46-1B-v0.1 \
  -q --q-bits 4 --mlx-path MiLMMT-46-1B-v0.1-4bit

Converted with mlx-lm 0.31.3.

License

Gemma Terms of Use, inherited from the base model — not Apache 2.0. MiLMMT-46-1B is derived from Gemma 3, so redistribution and use are governed by the Gemma Terms and the Gemma Prohibited Use Policy, which pass through to anyone who obtains these weights.

Citation

@misc{shang2026scalingmodeldatamultilingual,
      title={Scaling Model and Data for Multilingual Machine Translation with Open Large Language Models},
      author={Yuzhe Shang and Pengzhi Gao and Wei Liu and Jian Luan and Jinsong Su},
      year={2026},
      eprint={2602.11961},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2602.11961},
}