Qwen3.8-Flash-Next-Uncensored-oQ6e-mtp

Qwen3.8-Flash-Next-Uncensored (MLX, oQ6e, MTP)

Calibrated 6-bit mixed-precision MLX quantization of orcarouter/Qwen3.8-Flash-Next-Uncensored, the BF16 abliterated build of Qwen/Qwen3.8-Flash-Next. Produced with oMLX oQe (oQ level 6 with imatrix weighting). 6.7 bits per weight effective, 150.45 GB on disk (140.1 GiB) across 28 safetensors shards. Every part of the source is kept: routed experts at 6-bit, the 51B-parameter n-gram embedding table (quantized), the Lightning MTP head, and the full vision + video tower in BF16.

Who this is for. 6-bit experts are as close to the BF16 source as oQ goes before the 8-bit level, so this is the quality-first build. On a 192 GB or larger Apple Silicon Mac it runs fully resident with the full 262,144-token context and MTP. On a 128 GB Mac it runs, but only with the n-gram table on SSD, Lightning MTP switched off, and prompts up to about 131k tokens. The exact recipe and the reasons are in the 128 GB section below, measured rather than estimated. If you need the full 262k window on 128 GB, use the sibling oQ5e-mtp build, which was verified at 250k tokens.

[!WARNING] The source model has had its safety alignment removed by abliteration. It will comply with requests the original Qwen3.8-Flash-Next refuses. Released for research, red-teaming and interpretability work. Read the source model card's disclaimer before use and add your own moderation if you deploy it. You are responsible for what you do with it.

Model details

What is in this repo

The size shown by oMLX's estimator and in its upload dialog (140.1 "GB") is GiB. The decimal figure is 150.5 GB.

Quantization layout

oQ is sensitivity-driven mixed precision: every layer's quantization error is measured on calibration data, and bits go where the data says they matter. The "e" (oQe) adds an importance matrix so that inside each quantization group the clipping favors the weights that carry activation energy. Output is plain affine quantization that any MLX runtime can read.

Bit histogram over the quantized text tensors: 219 at 8-bit, 164 at 6-bit, plus 128 n-gram shards at 6-bit. The exact per-tensor map is in config.json under quantization.

How it was made

Built on a MacBook Pro M5 Max (128 GB) with oMLX 0.7.0.dev2 from the 360 GB BF16 source, streamed tensor by tensor (the full model never sits in memory).

• Sensitivity pass: oMLX's automatic RAM-safe path: it built a temporary uniform 4-bit proxy of the model (98 GB on disk, 66 GB resident, deleted afterwards) and scored all 48 layers on it with 128 samples x 256 tokens. A uniform probe was chosen deliberately over an existing mixed-precision quant, because oQ's probe perturbs each layer one bit below its own width and only ranks layers fairly when they all start at the same width.
• Importance matrix: reused from the oQ5e build's collection (the cache is keyed by source, calibration set and sample counts, not bit width). Calibration set oqe_code_multilingual (code, English, Korean, Chinese, Japanese, tool calls, reasoning), 1,024 samples x 512 tokens, 937 tensor entries, expert coverage 75,243 of 75,264 (21 experts never routed on the calibration set fall back to plain oQ). Three tensors have no imatrix entry by design (lm_head and the two hyper-connection mixer projections). Full report in oq_imatrix_report.json.
• Time: 12.7 minutes for the quantization itself with the cached imatrix, about 16 minutes including the proxy build and sensitivity pass.

Requirements

Apple Silicon only. Tested with oMLX 0.7.0.dev2, which ships its own qwen4_exp implementation (Lightning MTP, the PLE table on SSD, TurboQuant KV, MoE expert offload). The checkpoint is standard MLX safetensors, but stock mlx-lm / mlx-vlm need qwen4_exp support to load it and were not tested here.

Disk: 151 GB for the model, plus room for oMLX's KV cache spillover.

Running it on a 128 GB MacBook Pro (M5 Max), what works and what does not

All of this was measured on a MacBook Pro M5 Max, 128 GB, 40-core GPU, oMLX 0.7.0.dev2. The numbers oMLX logs are GiB.

The constraint. With the n-gram table on SSD, the resident weights are 103 GiB with the MTP head loaded, 101 GiB without it. oMLX's prefill guard caps a request at 95 percent of the Metal wired-memory limit (121.6 GiB on this machine, so 115.5 GiB). During a long prefill the process footprint grows with context: about 27 KB of KV per token plus transient attention buffers that Metal releases lazily. By 133k tokens of context the footprint reaches roughly 114 GiB, and the guard refuses the next chunk. That happened at exactly the same KV length in every attempt (133,120 tokens), with MTP on or off.

Settings that work on 128 GB:

Measured at 131,072 prompt tokens with those settings: prefill 974 tok/s, first token after 134 s, decode 25.1 tok/s (MTP off, at 131k context; short-context decode without MTP was not measured on this build). The guard did not throttle; every chunk ran at the full 2,048-token step.

Things that were tried and do not help, so you can skip them:

• Re-quantizing the MTP head harder. The whole head is 2.12 GB, and 1.89 GB of it is the three 512-expert tensors. Going from 6-bit to 4-bit would save 0.63 GB; to 3-bit about 0.9 GB. The gap at 133k is larger than that, so no MTP quant reaches even 133k, let alone 262k. Dropping the head entirely (MTP off) is the only MTP change that moves the needle.
• MoE expert offload (streaming a slice of the experts from SSD). At 90 percent resident it frees about 10 GB, enough to pass the guard, but prefill collapsed to 23 to 58 seconds per 2,048-token chunk (35 to 90 tok/s). A 200k prompt would wait over an hour for its first token. oMLX also refuses to combine expert offload with Lightning MTP.
• Raising the Metal wired limit (sudo sysctl iogpu.wired_limit_mb=127500, 97.5 percent of RAM) adds about 2.8 GiB to the cap. That is worth perhaps 20k more tokens of context, not 262k, and oMLX's own code comments warn that wiring above roughly 95 percent of RAM invites a jetsam that strands GPU memory until reboot. Not recommended.

If you want the full 262k window on 128 GB, use oQ5e-mtp. It is 22 GB smaller, keeps MTP on, and completed a real 250,073-token request in 214 s on the same machine.

Running it with oMLX (192 GB and up)

Drop the folder into ~/.omlx/models/ (or download it there). Recommended settings: Lightning MTP on with 3 draft tokens, TurboQuant KV 4-bit optional, context window 262144, thinking on, vendor sampling defaults (thinking: temperature 1.0, top_p 0.95, top_k 20, presence 0; instruct: 0.7 / 0.8 / 20 / presence 1.5). Any OpenAI-compatible client then works against http://localhost:8000/v1. Vision input goes in user messages as image_url parts.

Sibling builds

Limitations

• Apple Silicon and MLX only.
• Safety alignment removed by the source model's abliteration. No guardrails.
• On 128 GB machines: MTP unavailable in practice and context capped near 131k, as described above.
• 21 of 75,264 routed experts and 3 shell tensors were quantized without imatrix weighting (plain oQ).
• No formal perplexity or benchmark evaluation was run on this quant. The figures above are throughput measurements and a load/serve verification.

Credits

• Base model: Qwen (Qwen3.8-Flash-Next).
• Abliterated source: OrcaRouter.
• Quantization tooling: oMLX by jundot, oQ specification.

@misc{omlx,
  title  = {oMLX: LLM inference optimized for Apple Silicon},
  author = {jundot},
  year   = {2026},
  url    = {https://github.com/jundot/omlx}
}