Qwen3.6-35B-A3B-PRISM-MLX-NVFP4

Qwen3.6-35B-A3B-PRISM-MLX-NVFP4

Apple Silicon / MLX port of Ex0bit/Qwen3.6-35B-A3B-PRISM-NVFP4.

PRISM softens over-refusal and removes bias / propaganda patterns while keeping task quality, coherence, and multimodal ability. This repo is the MLX-native NVFP4 checkpoint for oMLX on Mac.

Sibling GPU repo (vLLM / compressed-tensors):
Ex0bit/Qwen3.6-35B-A3B-PRISM-NVFP4

Collections (browse the full family like Unsloth’s pages):

---

One-click setup (recommended)

Use this on a new Mac. It installs stock oMLX, downloads this model, installs the required NVFP4 compat patch, writes launchers, and starts the server. Do not use plain omlx serve for this checkpoint (see below).

Requirements

Option A — curl (easiest, no clone)

Open Terminal and run:

curl -fsSL https://huggingface.co/Ex0bit/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4/resolve/main/scripts/setup_one_click.sh | bash

First run can take a while (Homebrew/oMLX + ~25 GB model download). Leave the window open until it prints Done.

Option B — after you already downloaded the repo

# e.g. hf download Ex0bit/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4 --local-dir ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4
cd ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4
bash scripts/setup_one_click.sh

If model.safetensors is already under ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4, the script skips re-downloading.

What the script does

1. Checks Apple Silicon + unified memory
2. Installs/upgrades latest stock oMLX (Homebrew, else pipx)
3. Ensures Hugging Face CLI (hf / huggingface-cli)
4. Downloads this model to ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4
5. Installs patches/nvfp4_compat.py into the oMLX Python env (.pth + runtime check)
6. Writes ~/.omlx/settings.json + model preset (sampling + MTP enabled)
7. Installs launchers (compat-first — not bare omlx serve):
   • ~/.omlx/bin/start-prism-nvfp4.sh
   • Desktop: Start PRISM NVFP4.command
8. Optionally runs scripts/verify_e2e.py
9. Starts the server and opens the chat UI

When finished

Recommended sampling (preset defaults):

temperature = 0.6
top_p       = 0.95
top_k       = 20

Optional environment variables

Set these before running the script:

Examples:

# Custom port + skip the long e2e check
OMLX_PORT=10000 SKIP_VERIFY=1 \
  curl -fsSL https://huggingface.co/Ex0bit/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4/resolve/main/scripts/setup_one_click.sh | bash

# Model already on an external drive
MODEL_DIR=/Volumes/Models bash scripts/setup_one_click.sh

After install — daily use

# Start / restart (always use this — not plain omlx serve)
~/.omlx/bin/start-prism-nvfp4.sh

# Or open Desktop → Start PRISM NVFP4.command

API smoke:

curl -s http://127.0.0.1:9999/v1/models | head
curl http://127.0.0.1:9999/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen3.6-35B-A3B-PRISM-MLX-NVFP4",
    "messages": [{"role":"user","content":"Say only: HELLO_WORLD"}],
    "temperature": 0,
    "max_tokens": 16
  }'

Troubleshooting one-click

Important: always start via the one-click launcher. Stock omlx serve alone rejects this checkpoint’s weight_global_scale tensors.

---

Will this run on the latest oMLX from GitHub?

Yes — on Apple Silicon — if you use this repo’s one-click or start_omlx_nvfp4.py.
Plain omlx serve without the companion patch does not load this checkpoint.

Verified path (zero → generate): remove patch → load fails → nvfp4_compat install → load (280 global-scale wrappers) → greedy HELLO_WORLD.
Weights are a lossless MTP-key repack (mtp.*); tensor bytes match the calibrated NVFP4 conversion.

Re-run the harness on your machine:

# same Python that runs omlx:
python scripts/verify_e2e.py --model-dir ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4

Why a patch is required (not a second model)

The NVFP4 conversion keeps per-tensor weight_global_scale.
Metal MLX QuantizedLinear only applies per-group FP8 scales. Folding g into FP8 at convert time hurts fidelity (and can underflow). This release keeps globals bit-exact and applies 1/g after the matmul via a thin portable wrapper: patches/nvfp4_compat.py.

That interceptor is not in upstream jundot/omlx as of the verified 0.5.x line. One-click always starts through scripts/start_omlx_nvfp4.py so the patch is active before weights load.

Lightning MTP: checkpoint uses stock mtp.* names; enable mtp_enabled in oMLX model settings (preset ships enabled).

---

Manual install

1. oMLX

# App (easiest UI)
# https://github.com/jundot/omlx/releases

# or Homebrew
brew tap jundot/omlx https://github.com/jundot/omlx
brew install omlx

# or from source
git clone https://github.com/jundot/omlx.git && cd omlx && pip install -e .

2. Download weights

mkdir -p ~/Models
hf download Ex0bit/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4 \
  --local-dir ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4

3. NVFP4 compat (required)

cd ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4
# Use the same Python that runs omlx:
python patches/nvfp4_compat.py install

4. Serve

python scripts/start_omlx_nvfp4.py serve --model-dir ~/Models --port 9999
# open http://127.0.0.1:9999/admin/chat

Point oMLX’s model directory at ~/Models (Welcome wizard or Settings). Pin Qwen3.6-35B-A3B-PRISM-MLX-NVFP4 as default if you want it always loaded.

5. API example

curl http://127.0.0.1:9999/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen3.6-35B-A3B-PRISM-MLX-NVFP4",
    "messages": [{"role":"user","content":"In one sentence, what is mixture-of-experts?"}],
    "temperature": 0.6,
    "top_p": 0.95,
    "max_tokens": 256
  }'

---

Files

---

Hardware guidance

---

How this was produced

1. Start from Ex0bit/Qwen3.6-35B-A3B-PRISM-NVFP4 (compressed-tensors NVFP4).
2. Repack packed U8 weights → MLX U32 NVFP4 layout; keep FP8 group scales and weight_global_scale; rename MTP mtp.* → mtp_head.*; keep vision BF16.
3. Serve under oMLX VLM engine (model_type: qwen3_5_moe) with the global-scale interceptor so Metal matmul stays bit-faithful to the calibrated quant.

---

Reference machine layout (how the author runs it)

Start oMLX.app  (Desktop)
  └─ do shell script ~/.omlx/bin/start-omlx.sh
       └─ ~/.local/bin/omlx serve          # port 9999
            model_dirs: ~/Models
            default+pinned:
              ~/Models/Qwen3.6-35B-A3B-PRISM-MLX-NVFP4

That process uses a local oMLX tree with in-package NVFP4 patches. The files in this HF repo make the same behaviour available on stock oMLX via the compat layer + launcher.

---

License

Apache 2.0 (inherited from the base model and PRISM NVFP4 release).

<div align="center">

Support

</div>