MiLMMT-46-4B-v1.0-6bit-MLX

MiLMMT-46-4B v1.0 — 6-bit (MLX, text-only)

A 6-bit, group-size-64 MLX quantization of Xiaomi's MiLMMT-46-4B-v1.0, for on-device translation on Apple Silicon. 46 languages.

The base checkpoint is a Gemma3ForConditionalGeneration with a SigLIP vision tower. This conversion is text-only: vision_tower and multi_modal_projector are dropped, leaving the language model — 8.6 GB (bf16, with vision) down to 3.0 GB.

Which build should you use?

Most people should take the 5-bit build, not this one.

The ladder was measured against the unquantized bf16 model on FLORES+ devtest, en→X, 45 languages × 100 sentences (4,500 segments per rung), chrF++ against gold human references, with sacrebleu's paired bootstrap (2,000 resamples) for significance:

5-bit already measures indistinguishable from full precision (Δ −0.04, 95% CI ±0.43, and a 23/45 language split that is a coin flip). Quantization error is monotone in bit-width, so 6-bit cannot be closer to bf16 than "also indistinguishable" — it costs 480 MB more than 5-bit for no gain this benchmark can measure.

This rung was not benchmarked directly. Its position above the floor is an inference from the 5-bit measurement plus monotonicity, not a measurement of these weights. It is published for completeness — for anyone who wants extra margin on a workload unlike FLORES+ news text, or who is validating quantization behaviour across the ladder.

Usage

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load("translate-studio/MiLMMT-46-4B-v1.0-6bit-MLX")

src, tgt = "Chinese (Simplified)", "English"
text = "我爱机器翻译"
prompt = f"Translate this from {src} to {tgt}:\n{src}: {text}\n{tgt}:"

print(generate(model, tokenizer, prompt=prompt, max_tokens=512,
               sampler=make_sampler(temp=0.0), verbose=False))

Prompt format

The model card's plain block, with no BOS token and no chat template — the checkpoint's chat_template.jinja just concatenates message contents, so there are no turn markers:

Translate this from <source language name> to <target language name>:
<source language name>: <source sentence>
<target language name>:

Language names must be written as the base model card lists them (e.g. Chinese (Simplified), Chinese (Traditional), English).

Note on stop tokens

config.json carries only eos_token_id: 1, while generation_config.json lists [1, 106] (106 is <end_of_turn>). Loaders that read the stop id from config.json alone can miss 106 and run on to max_tokens. Honour both.

Supported languages

Arabic, Azerbaijani, Bulgarian, Bengali, Catalan, Czech, Danish, German, Greek, English, Spanish, Persian, Finnish, French, Hebrew, Hindi, Croatian, Hungarian, Indonesian, Italian, Japanese, Kazakh, Khmer, Korean, Lao, Malay, Burmese, Norwegian, Dutch, Polish, Portuguese, Romanian, Russian, Slovak, Slovenian, Swedish, Tamil, Thai, Tagalog, Turkish, Urdu, Uzbek, Vietnamese, Cantonese, Chinese (Simplified), Chinese (Traditional).

Prompts targeting Telugu, Marathi or Gujarati — which are not in that list — return fluent Hindi/Devanagari rather than an error. Filter to the supported set; do not rely on the model to refuse.

License & attribution

Derived from Gemma and distributed under the Gemma Terms of Use.

Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms.

Credit: Xiaomi Inc. (MiLMMT-46) and Google (Gemma 3 base architecture). This repository only provides an MLX-quantized, text-only copy of the released weights, redistributed under the same terms. Use restrictions in the Gemma Terms apply, including the prohibited-use policy.

@misc{han2026referencefreeposttrainingopenlarge,
      title={Reference-Free Post-Training of Open Large Language Models for Multilingual Machine Translation},
      author={Chris Han and Pengzhi Gao and Pei Fu and Jian Luan},
      year={2026},
      eprint={2608.10812},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2608.10812}
}