granite-4.1-3b-mxfp8-mlx

granite-4.1-3b-mxfp8-mlx

Brainwaves

         arc   arc/e boolq hswag obkqa piqa  wino
mxfp8    0.406,0.581,0.821,0.484,0.434,0.712,0.559

Other models in this size

gemma-4-E2B-it

mxfp8    0.376,0.464,0.743,0.490,0.378,0.709,0.622
mxfp4    0.380,0.451,0.762,0.494,0.374,0.699,0.594
q8-hi    0.392,0.462,0.762,0.487,0.376,0.706,0.636

Qwen3.5-4B

mxfp8    0.392,0.441,0.627,0.601,0.360,0.739,0.590
mxfp4    0.371,0.444,0.632,0.585,0.356,0.732,0.548
q8-hi    0.398,0.435,0.622,0.604,0.362,0.732,0.585
q8       0.398,0.434,0.622,0.604,0.362,0.733,0.582

This model granite-4.1-3b-mxfp8-mlx was converted to MLX format from ibm-granite/granite-4.1-3b using mlx-lm version 0.31.3.

Use with mlx

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("granite-4.1-3b-mxfp8-mlx")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True, return_dict=False,
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)