Apertus-v1.5-8B-text-MLX-mxfp8

m1rkocasu/Apertus-v1.5-8B-text-MLX-mxfp8

Apertus-v1.5-8B-text-MLX-mxfp8 is a text generation model from m1rkocasu. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as apache-2.0.

<h1 align="center"Apertus v1.5 8B text — MLX MXFP8</h1

Downloads · 30 days

33% of all-time downloads

All-time downloads

116

Public

Parameters

8.1B

8.3 GB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors8.3 GB · 100%

Parameter types

How the weights are stored.

U328.1B · 100%

Try a prompt

Base models

swiss-ai/Apertus-v1.5-8B

Model card

From the Hugging Face model README

<h1 align="center">Apertus v1.5 8B text — MLX MXFP8</h1> <p align="center"> <a href="https://huggingface.co/swiss-ai/Apertus-v1.5-8B">Base model</a> | <a href="https://apertus-ai.org/">Apertus</a> | <a href="https://github.com/ml-explore/mlx-lm">mlx-lm</a> | <a href="https://github.com/jundot/omlx">oMLX</a> | <a href="#the-family">The family</a> <br> <b>Format</b>: MLX | <b>Weights</b>: MXFP8, 8.31 GB | <b>License</b>: <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache 2.0</a> </p>

This repository holds the text branch of Apertus 1.5 8B, converted to MLX and quantized to MXFP8. It runs on Apple silicon through mlx-lm.

Apertus 1.5 is the fully open model of the Swiss AI Initiative, built at EPFL, ETH Zurich and the Swiss National Supercomputing Centre on open data.

This conversion was made independently of the Apertus release.

Model summary

The family

Six builds of the same text branch, measured on the same corpus with the same script. Perplexity is measured on the test split of Salesforce/wikitext, configuration wikitext-2-raw-v1, over 200 non-overlapping windows of 512 tokens, which is 102,400 scored tokens.

<div align="center" style="overflow-x:auto"> <table style="border-collapse:collapse;margin:0 auto"> <thead><tr><th style="padding:9px 14px;border-bottom:2px solid #45BED9;color:#45BED9;font-weight:600;font-size:13px;text-align:left">Build</th><th style="padding:9px 14px;border-bottom:2px solid #45BED9;color:#45BED9;font-weight:600;font-size:13px;text-align:center">Bits per weight</th><th style="padding:9px 14px;border-bottom:2px solid #45BED9;color:#45BED9;font-weight:600;font-size:13px;text-align:center">Size</th><th style="padding:9px 14px;border-bottom:2px solid #45BED9;color:#45BED9;font-weight:600;font-size:13px;text-align:center">Perplexity</th><th style="padding:9px 14px;border-bottom:2px solid #45BED9;color:#45BED9;font-weight:600;font-size:13px;text-align:center">Against the 8-bit</th><th style="padding:9px 14px;border-bottom:2px solid #45BED9;color:#45BED9;font-weight:600;font-size:13px;text-align:left">Use it when</th></tr></thead> <tbody> <tr><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left"><a href="https://huggingface.co/m1rkocasu/Apertus-v1.5-8B-text-MLX-8bit">8-bit</a></td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">8.500</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">8.54 GB</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">10.4088</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">reference</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">quality first, and the reference every other build here is measured against.</td></tr> <tr><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left"><strong>MXFP8</strong> *(this repository)*</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">8.250</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">8.31 GB</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">10.4748</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">+0.6%</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">you want the floating-point format. In perplexity it sits next to the 6-bit, which is smaller.</td></tr> <tr><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left"><a href="https://huggingface.co/m1rkocasu/Apertus-v1.5-8B-text-MLX-6bit">6-bit</a></td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">6.500</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">6.54 GB</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">10.4618</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">+0.5%</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">two GB less than the 8-bit for half a percent of perplexity.</td></tr> <tr><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left"><a href="https://huggingface.co/m1rkocasu/Apertus-v1.5-8B-text-MLX-5bit">5-bit</a></td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">5.500</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">5.54 GB</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">10.5261</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">+1.1%</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">three GB less than the 8-bit, and still within one and a half percent of it.</td></tr> <tr><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left"><a href="https://huggingface.co/m1rkocasu/Apertus-v1.5-8B-text-MLX-4bit-DWQ">4-bit DWQ</a></td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">4.500</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">4.53 GB</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">10.9691</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">+5.4%</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">the smallest build that stays close: distillation buys back most of what plain 4-bit gives away, at the same file size.</td></tr> <tr><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left"><a href="https://huggingface.co/m1rkocasu/Apertus-v1.5-8B-text-MLX-mxfp4">MXFP4</a></td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">4.250</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">4.28 GB</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">10.8522</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">+4.3%</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">the smallest build here, and now calibrated: closer to the 8-bit than the 4-bit DWQ, a quarter of a GB lighter.</td></tr> </tbody> </table> </div> <p style="margin-top:12px;font-size:11px;opacity:0.7"> * The 8-bit build is the reference because a bfloat16 run does not fit the 24 GB machine these were made on: its 15 GB of weights page to disk. The step from 8 bits to 6 costs half a percent, which makes a large gap between bfloat16 and 8 bits unlikely. That is an inference, not a measurement.<br> * Perplexity compares quantizations of one model on one corpus. It says nothing about how this model compares to a different model, and nothing about how well it follows instructions. </p> <img src="https://huggingface.co/m1rkocasu/Apertus-v1.5-8B-text-MLX-mxfp8/resolve/main/assets/perplexity-vs-size.png" alt="Perplexity against size on disk for the six text builds; the MXFP8 build is highlighted" width="780">

What was left out

These variants were measured on the same corpus and are not published. The table is here so that nobody repeats the work.

<div align="center" style="overflow-x:auto"> <table style="border-collapse:collapse;margin:0 auto"> <thead><tr><th style="padding:9px 14px;border-bottom:2px solid #45BED9;color:#45BED9;font-weight:600;font-size:13px;text-align:left">Variant</th><th style="padding:9px 14px;border-bottom:2px solid #45BED9;color:#45BED9;font-weight:600;font-size:13px;text-align:center">Bits per weight</th><th style="padding:9px 14px;border-bottom:2px solid #45BED9;color:#45BED9;font-weight:600;font-size:13px;text-align:center">Size</th><th style="padding:9px 14px;border-bottom:2px solid #45BED9;color:#45BED9;font-weight:600;font-size:13px;text-align:center">Perplexity</th><th style="padding:9px 14px;border-bottom:2px solid #45BED9;color:#45BED9;font-weight:600;font-size:13px;text-align:left">Why it is not published</th></tr></thead> <tbody> <tr><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">MXFP4, rounded to nearest (published here until 2026-09-19)</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">4.250</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">4.28 GB</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">11.71</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">replaced by the calibrated MXFP4, same size</td></tr> <tr><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">mixed 4/6, affine</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">5.000</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">5.03 GB</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">11.24</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">bigger than the 4-bit DWQ build, and weaker</td></tr> <tr><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">4-bit, affine, no distillation</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">4.500</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">4.53 GB</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">11.59</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">same size as the 4-bit DWQ build, and weaker</td></tr> <tr><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">3-bit, group size 32</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">4.000</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">4.03 GB</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">66.06</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">unusable</td></tr> <tr><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">3-bit DWQ, group size 64</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">3.500</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">3.52 GB</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">50.91</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">unusable</td></tr> <tr><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">mixed 3/6, affine</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">4.250</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">4.28 GB</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">140.93</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">unusable</td></tr> <tr><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">3-bit, group size 64</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">3.500</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">3.52 GB</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:center">169.79</td><td style="padding:8px 14px;border-bottom:1px solid rgba(128,128,128,0.25);font-size:14px;text-align:left">unusable</td></tr> </tbody> </table> </div>

Three bits break this model. At a group size of 64 it answers fluently and wrongly, completing "The capital of Switzerland is" with "not a good idea"; at 32 it answers "Bern" and still reaches a perplexity of 66. Distillation recovers a large share of the loss, from 169.79 to 50.91, and the result is still five times the reference.

The two four-bit rows are here for a different reason: they work, but each is beaten by a published build that is the same size or smaller. Nothing between 4.25 and 4.5 bits per weight was worth publishing besides the two that are.

<div style="border-left:4px solid #45BED9;background:rgba(69,190,217,0.07);border-radius:0 4px 4px 0;padding:12px 16px;margin:14px 0"> <p style="margin:0"><strong>Need images or audio?</strong> The whole model, this same decoder plus the vision and audio towers in float32, is a separate family, converted to MLX for mlx-vlm. Start at <a href="https://huggingface.co/m1rkocasu/Apertus-v1.5-8B-MLX-8bit">Apertus v1.5 8B MLX</a>.</p> </div>

This build

MXFP8 stores each weight as an 8-bit float (E4M3) and shares one 8-bit power-of-two exponent across every group of 32, which is exactly what mlx_lm.convert --q-mode mxfp8 writes; there is no calibration. An 8-bit float is not an 8-bit integer: E4M3 keeps three bits of mantissa, so every weight is rounded to within about 6% of its own size, where the affine 8-bit spreads 256 even steps across each group of 64. Measured on this model's weights, that is about 24 times the squared error of the affine 8-bit and one and a half times the 6-bit's, and the perplexity lands next to the 6-bit's.

Build MXFP8 with mlx 0.32 or later. The MXFP8 quantizer of mlx 0.31.2 writes wrong values, about eight times the error of E4M3 (checked against mlx 0.32.2 and against PyTorch's E4M3): built that way this repository measured 11.02 instead of 10.47. Reading is not affected, so this build loads correctly with either version.

Its perplexity is 10.4748, +0.6% against the 8-bit build, on the protocol described above.

mlx_lm.convert --hf-path <text branch> \
  --mlx-path Apertus-v1.5-8B-text-MLX-mxfp8 -q --q-mode mxfp8

Built and measured on 2026-09-19 with mlx 0.32.2 and mlx-lm 0.31.3 on macOS 27.0.

Run it

pip install mlx-lm
mlx_lm.generate --model m1rkocasu/Apertus-v1.5-8B-text-MLX-mxfp8 \
  --prompt "Name the capital of Switzerland and say one sentence about it." \
  --max-tokens 200

From Python:

from mlx_lm import load, generate

model, tokenizer = load("m1rkocasu/Apertus-v1.5-8B-text-MLX-mxfp8")
messages = [{"role": "user", "content": "Name the capital of Switzerland."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=200))

Behind an OpenAI-compatible endpoint:

mlx_lm.server --model m1rkocasu/Apertus-v1.5-8B-text-MLX-mxfp8

In oMLX, search for m1rkocasu/Apertus-v1.5-8B-text-MLX-mxfp8 and download it. Deliberation is read natively: the model opens it with <think> and closes it with </think>, which is what oMLX looks for.

What the text branch is

The upstream release is multimodal. It reads images and audio as well as text, and its architecture is Apertus1p5ForConditionalGeneration, which neither mlx-lm 0.31.3 nor mlx-vlm 0.6.17 implements.

This repository holds the text decoder on its own, declared as ApertusForCausalLM so that mlx-lm loads it. The token embedding goes from 266752 rows to 131072. The rows that go are the image and audio codebooks, which start at index 131272 and 262344 in the upstream vocabulary. The upstream output_vocab_size is already 131072, so the language modelling head is untouched and no text token is lost.

The model reads and writes text. It does not accept images or audio. The whole model, towers included, is a different family: Apertus v1.5 8B MLX.

Limitations

The quantization inherits every limitation of the upstream model, which its model card describes. No output filter ships with these weights.

Quality here is measured by perplexity on one English corpus. Apertus 1.5 is multilingual, and the effect of quantization on languages other than English is not measured in this collection. Instruction following, reasoning and tool use are also unmeasured.

License and acceptable use

The weights stay under the Apache 2.0 license of the upstream release. Use is also subject to the Apertus 1.5 acceptable use policy and privacy policy:

For removal of personal or copyrighted data, write to the Swiss AI Initiative at [email protected] or [email protected].

Credits

The model is the work of the Swiss AI Initiative. This repository adds the MLX conversion, the quantization and the measurements above.

@misc{ApertusV15,
  author       = {{Swiss AI Initiative}},
  title        = {Apertus v1.5},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/swiss-ai/Apertus-v1.5-8B}},
  note         = {EPFL, ETH Zurich, and the Swiss National Supercomputing Centre}
}

Task: Text Generation
Library: mlx
Type: apertus
License: apache-2.0
Created: Sep 19, 2026
Updated: Sep 19, 2026