granite-4.2-8b-mxfp8-mlx

granite-4.2-8b-mxfp8-mlx

Brainwaves

         arc   arc/e boolq hswag obkqa piqa  wino
q8-hi    0.472
mxfp8    0.483,0.653,0.862,0.552,0.410,0.734,0.594

Previous model

granite-4.1-8b-Abliterated-AND-Disinhibited

         arc   arc/e boolq hswag obkqa piqa  wino
mxfp8    0.496,0.692,0.864,0.666,0.466,0.770,0.632
q8-hi    0.477,0.658,0.865,0.657,0.460,0.755,0.604
q8       0.480,0.657,0.861,0.656,0.456,0.755,0.609

granite-4.1-8b

         arc   arc/e boolq hswag obkqa piqa  wino
mxfp8    0.486,0.666,0.875,0.636,0.450,0.766,0.631

The performance in q8-hi dips below the old base, even if generally int quants underperform in this model.

The mxfp8 shows the most degradation.

Model abliteration in the previous version shows a dramatic increase in performance above base.

More metrics coming soon.

-G

This model granite-4.2-8b-mxfp8-mlx was converted to MLX format from ibm-granite/granite-4.2-8b using mlx-lm version 0.31.3.

Use with mlx

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("granite-4.2-8b-mxfp8-mlx")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True, return_dict=False,
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)