Qwen3.8-27B-Abliterated-MLX

Qwen3.8-27B Abliterated MLX

An unofficial family of MLX derivatives of Qwen/Qwen3.8-27B. The original model is by Qwen; the MLX conversions, refusal-direction experiment, and validation were performed by PocketAI Model Lab. PocketAiHub identifies the publisher of these derivatives, not the creator of Qwen3.8.

This consolidated repository contains experimental 2-bit AWQ plus 4-bit, 6-bit, 8-bit, and BF16 variants. It is the canonical download location.

Important safety notice

These checkpoints have been modified to suppress learned refusal behavior. They may produce harmful, illegal, offensive, deceptive, or dangerously incorrect content more readily than the upstream instruction model. Abliteration is not truthfulness training, capability improvement, or a safety guarantee. Use these models only where you can independently evaluate and constrain their outputs.

Variants

All variants share the pinned upstream revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. Quantized variants use MLX affine quantization; 2-bit uses AWQ/group 32 and 4/6/8-bit use group 64. Their vision towers remain in BF16.

Abliteration recipe

A projected harmful-minus-harmless direction was measured from 256 length-matched prompts per class at the assistant-generation boundary.

• Direction source layer: 53
• Destination layers: 24–63
• Scale: 1.0
• Per-input-column norm preservation: enabled
• Modified residual-output matrices: 80

See abliteration-manifest.json for the machine-readable recipe.

Behavioral evaluation

Each variant was evaluated deterministically on 100 harmful prompts and 100 benign controls with batch size 1, thinking disabled, and a 128-token ceiling.

The scorer is a transparent phrase-based explicit-refusal detector. Most generations reached the token ceiling, so this is an early-refusal screen rather than a full-completion evaluation. It cannot establish universal compliance or answer quality. Machine-readable results are in benchmarks/validation-summary.json and each variant's validation-summary.json.

Feature validation

The 4-bit, 6-bit, 8-bit, and BF16 variants passed:

• 12/12 deterministic quality checks
• 8/8 native tool-call checks
• text and image smoke tests
• temporal video understanding (red->blue)
• 4K-context retrieval (COBALT-7319)

The 2-bit AWQ build is experimental. It measured 9/12 quality checks, 0/8 tool-call checks, passed vision but not exact-text smoke, missed the synthetic temporal-video check, and found the 4K needle but did not return it exactly. These limitations are preserved in the machine-readable evidence.

4K MLX performance

These are single local runs on an Apple M5 Max with 128 GB unified memory, using mlx==0.32.0, mlx-vlm==0.6.8, batch size 1, temperature 0, seed 0, and thinking disabled. Each prompt contained 4,105 tokens.

These measurements are not cross-machine performance guarantees.

Download and load

python -m pip install "mlx==0.32.0" "mlx-vlm==0.6.8"

Download only the desired variant:

from pathlib import Path

from huggingface_hub import snapshot_download
from mlx_vlm import generate, load
from mlx_vlm.prompt_utils import apply_chat_template

repo_id = "PocketAiHub/Qwen3.8-27B-Abliterated-MLX"
variant = "4bit"  # "2bit", "4bit", "6bit", "8bit", or "bf16"

snapshot = Path(snapshot_download(repo_id, allow_patterns=[f"{variant}/*"]))
model, processor = load(str(snapshot / variant))

prompt = apply_chat_template(
    processor,
    model.config,
    "Explain why seasons occur.",
    num_images=0,
    enable_thinking=False,
)
result = generate(
    model,
    processor,
    prompt,
    max_tokens=256,
    temperature=0.0,
    enable_thinking=False,
)
print(result.text)

Image and video inputs use the normal mlx_vlm.generate media arguments.

License and attribution

The base model is Apache 2.0 licensed. See LICENSE and the official Qwen model card.