gemma-4-12B-coder-fable5-composer2.5-v1-mxfp8-mlx

gemma-4-12B-coder-fable5-composer2.5-v1-mxfp8-mlx

Brainwaves

         arc   arc/e boolq hswag obkqa piqa  wino
mxfp8    0.422,0.549,0.754,0.537,0.388,0.717,0.611

Quant    Perplexity      Peak Memory   Tokens/sec
mxfp8   292.901 ± 5.389  18.27 GB      330

Baseline model

google/gemma-4-12B-it

         arc   arc/e boolq hswag obkqa piqa  wino
mxfp8    0.385,0.527,0.766,0.509,0.386,0.664,0.579

Quant    Perplexity      Peak Memory   Tokens/sec
mxfp8   175.766 ± 3.092  19.42 GB      463

Use with mlx

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("gemma-4-12B-coder-fable5-composer2.5-v1-mxfp8-mlx")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True, return_dict=False,
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)