Lizzy-7B-mlx-bf16

Lizzy-7B — MLX (bf16)

Native MLX port of Flower Labs' Lizzy-7B, a UK-built open-weight model released April 2026.

This is an unofficial community port, not affiliated with or endorsed by Flower Labs. No MLX support existed for this model before this. Flower Labs published their own GGUFs, but those require their own llama.cpp fork — the architecture was never upstreamed to mainline llama.cpp. This is the first MLX port.

Full-precision bf16, ~14GB. If you're on a memory-constrained Mac, see the 8-bit or 4-bit versions instead.

Why a from-scratch port

Lizzy's trust_remote_code implementation requires transformers>=5.4.0 — every release older than that breaks it (below 5.0, the tokenizer needs a TokenizersBackend class that doesn't exist yet; on 5.0-5.3, the model code calls Cache.get_mask_sizes() with the pre-5.4 convention and crashes at generation). It works fine from 5.4.0 through current. Rather than depend on trust_remote_code at all, this is a real architecture port: native MLX code, no fragile version pin.

Turns out Lizzy-7B is architecturally close to OLMo3 — post-norm decoder layout, full-width QK-norm (applied before the head reshape, not per-head), and YARN RoPE scaling on the 8 full-attention layers only (the 24 sliding-attention layers use plain RoPE). That's a more useful starting point than it sounds — most of the actual work was getting those specifics right, not inventing anything new.

Usage

lizzy isn't in stock mlx-lm yet (PR pending — ml-explore/mlx-lm). Until it lands, you need the registration shim to make mlx_lm recognize the architecture:

import importlib.util, sys
import mlx.core as mx

def register_lizzy(model_file="lizzy.py"):
    spec = importlib.util.spec_from_file_location("mlx_lm.models.lizzy", model_file)
    mod = importlib.util.module_from_spec(spec)
    sys.modules["mlx_lm.models.lizzy"] = mod
    spec.loader.exec_module(mod)

register_lizzy()  # lizzy.py from this repo, same directory or adjust the path

from mlx_lm import load, generate
model, tokenizer = load("Strikesure5555/Lizzy-7B-mlx-bf16")

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Hello"}], add_generation_prompt=True, tokenize=False
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=200))

Once the mlx-lm PR merges, the shim won't be necessary — mlx_lm.load will pick up the architecture natively.

Validation

Checked against the reference PyTorch implementation across fp32 layerwise parity, bf16 logit agreement, greedy-decode exact match, and KV-cache self-consistency (including negative controls — deliberately broke the implementation a few different ways to confirm the checks actually catch it, not just rubber-stamp). Full writeup in the source repo.

License

Apache-2.0, same as the base model.