Qwen3.8-27B-Uncensored-MLX-BF16

Qwen3.8-27B-Uncensored MLX BF16

Full-precision BF16 MLX conversion of orcarouter/Qwen3.8-27B-Uncensored, including native vision-language support and a separately packaged native MTP drafter.

This is a community conversion for Apple silicon. It is not an official Qwen or OrcaRouter release. No quantization, fine-tuning, pruning, or additional weight modification was applied during this conversion. The upstream model is an abliterated (refusal-direction removed) derivative of Qwen/Qwen3.8-27B; consult its model card for evaluation details and limitations.

What is included

The model has a 262,144-token configured context window. Real usable context depends on memory, KV-cache settings, prompt modality, and the MLX-VLM version.

Requirements

• Apple silicon Mac
• Current mlx-vlm with Qwen3.5/Qwen3.8 and MTP support
• Approximately 55 GB for weights, plus working memory and KV cache

One installation option:

uv tool install mlx-vlm --with jinja2

Text generation

mlx_vlm.generate \
  --model onchainengineer/Qwen3.8-27B-Uncensored-MLX-BF16 \
  --prompt "Explain why the sky is blue." \
  --thinking-mode enabled \
  --max-tokens 512

For deterministic output, add --temperature 0. For maximum response quality, leave weights and KV cache unquantized; long-context workloads may optionally trade fidelity for memory with MLX-VLM's KV-cache controls.

Native MTP speculative decoding

hf download onchainengineer/Qwen3.8-27B-Uncensored-MLX-BF16 \
  --local-dir ./Qwen3.8-27B-Uncensored-MLX-BF16

mlx_vlm.generate \
  --model ./Qwen3.8-27B-Uncensored-MLX-BF16 \
  --draft-model ./Qwen3.8-27B-Uncensored-MLX-BF16/mtp \
  --draft-kind mtp \
  --prompt "Write a clear technical explanation of speculative decoding." \
  --thinking-mode enabled \
  --max-tokens 512

Point --draft-model at the snapshot's mtp subdirectory. MTP can improve longer generations, but very short outputs may be slower because drafter setup dominates.

Vision-language generation

mlx_vlm.generate \
  --model onchainengineer/Qwen3.8-27B-Uncensored-MLX-BF16 \
  --image /absolute/path/to/image.png \
  --prompt "Describe this image precisely." \
  --max-tokens 256

Conversion provenance

• Source: orcarouter/Qwen3.8-27B-Uncensored
• Source revision: 9878936be9458522b5aeed0e13476bb8426f57f0
• Source inventory: 18 safetensors shards, 1,199 tensors, all BF16
• Converter: mlx-vlm 0.6.15
• MLX: 0.32.1
• Transformers: 5.15.1
• Conversion target: bfloat16, without quantization

Main conversion:

mlx_vlm.convert \
  --hf-path /path/to/orcarouter-Qwen3.8-27B-Uncensored \
  --mlx-path /path/to/Qwen3.8-27B-Uncensored-MLX-BF16 \
  --dtype bfloat16

MTP extraction:

python -m mlx_vlm.speculative.drafters.qwen3_5_mtp.split \
  --model /path/to/orcarouter-Qwen3.8-27B-Uncensored \
  --output /path/to/Qwen3.8-27B-Uncensored-MLX-BF16/mtp

Validation

The published artifact was checked locally on a 256 GB M3 Ultra Mac Studio:

• all 1,184 main tensors are BF16; all 15 MTP tensors are BF16
• no model quantization metadata is present
• deterministic text generation loaded and returned the expected answer
• reasoning-style generation loaded and produced a coherent mathematical explanation
• vision inference identified the subject and dominant colors in a test image
• native MTP loaded successfully and achieved 100% drafted-token acceptance on the deterministic smoke test
• measured peak unified memory was approximately 55.0 GB for text, 55.8 GB for vision, and 56.4 GB with MTP

End-to-end benchmarks

A reproducible 2026-08-22/23 campaign used MLX-VLM 0.6.15 and EvalScope 1.10.0 on an M3 Ultra (60-core GPU, 256 GB unified memory). Accuracy runs used temperature 0, thinking enabled, seed 42, and four-way continuous batching unless stated otherwise. Sampled results are explicitly labeled and are not full-leaderboard claims.

StrongREJECT generated all 313 target responses. The official rubric with a separate local Qwen3.5-35B-A3B-4bit judge scored 0.5907 (higher means more harmful/compliant), while the official package's string-matching baseline scored 0.7955. The official fine-tuned evaluator could not run because its google/gemma-2b dependency was gated for the benchmark account. Target generation had zero request errors; 157/313 responses reached the 1,024-token cap. Harmful prompts and raw responses are intentionally not published.

Serving highlights:

• cold process start: 11.11 s to HTTP ready, 12.02 s to first valid completion
• streaming TTFT: 0.329 s
• base decode: 13.03 tok/s; native MTP: 34.17 tok/s (2.62x)
• 128K needle retrieval passed in 669.76 s after setting MLX_VLM_TOKEN_QUEUE_TIMEOUT=3600
• one- and two-image inputs and tool calling passed
• JSON mode produced valid JSON but missed the requested key schema
• the exact model path worked with the server; a friendly model ID returned HTTP 400 and /v1/models omitted the preloaded LLM

LongBench v2 exposed real service limits: short scored 4/10, 9/10 medium samples completed (5/9 correct), one medium sample exceeded the 262,144-token context, and the sampled long split had no scored completions because nine documents exceeded context and the one in-window ~234K document produced no first token within 3,600 seconds.

See benchmark-results/2026-08-22/BENCHMARK_REPORT.md for confidence intervals, output-cap counts, per-category caveats, LongBench failure accounting, system measurements, and methodology. Machine-readable aggregates, exact EvalScope task configs, safe report JSON, system JSONL, and harness scripts are included alongside it.

Performance varies by hardware, prompt, software version, thermal state, and generation settings.

Safety and intended use

This checkpoint intentionally reduces refusal behavior. That does not make every generated answer accurate, safe, legal, private, or appropriate. Users are responsible for evaluating outputs and for complying with applicable law and the Apache-2.0 license. Do not use it to facilitate harm, unauthorized access, privacy violations, fraud, or other unlawful activity. Apply appropriate safeguards for any deployment exposed to other users.

License and attribution

Apache License 2.0. See LICENSE. This conversion retains attribution to Qwen and OrcaRouter; review the upstream model cards for provenance, training/modification details, evaluations, and known limitations.