Ornith-1.0-35B-heretic-mlx-8bit

Ornith-1.0-35B-heretic — MLX 8-bit

An 8-bit MLX quantization of thanet-s/Ornith-1.0-35B-heretic, optimized for Apple Silicon (LM Studio MLX engine, mlx-vlm).

The source is a full-weight Heretic 1.4.0 (abliterated / uncensored) derivative of deepreinforce-ai/Ornith-1.0-35B — DeepReinforce's state-of-the-art agentic-coding MoE. Per DeepReinforce, the Ornith-1.0 family is post-trained on top of Gemma 4 and Qwen 3.5 — but different variants use different bases (e.g. the family also ships a Gemma4-31B). This 35B-MoE loads under the qwen3_5_moe architecture class, and its config.json is entirely Qwen 3.5 MoE (no Gemma components), paired with a Qwen 3.5-style vision stack.

Model summary

Why MLX 8-bit

MLX runs natively on Apple Silicon's unified memory. At 8-bit this build keeps quality very close to the BF16 source while fitting comfortably on high-memory Macs. Because Ornith activates only ~3B parameters per token, throughput is far higher than a dense 35B model of the same footprint.

Recommended hardware: Apple Silicon Mac with 64 GB+ unified memory (128 GB gives ample headroom for long context). For 48 GB or less, prefer a 6-bit or 4-bit build.

Usage

LM Studio

1. Place this repo's folder under ~/.lmstudio/models/<your-username>/Ornith-1.0-35B-heretic-mlx-8bit/.
2. Select it in My Models, confirm the runtime is MLX, and load.
3. Start with a moderate context (16k–32k) and raise as needed.
4. Enable a Qwen3-compatible reasoning parser if you want <think> content hidden.

mlx-vlm (CLI)

pip install -U mlx-vlm
python -m mlx_vlm.generate \
  --model <your-username>/Ornith-1.0-35B-heretic-mlx-8bit \
  --max-tokens 256 \
  --prompt "Write a Python function that reverses a linked list."

Vision

This model includes a vision encoder. Pass an image with --image path/to/image.png when using mlx_vlm.generate.

Conversion

Converted from the BF16 source with mlx-vlm (8-bit, group size 64):

python -m mlx_vlm.convert \
  --hf-path thanet-s/Ornith-1.0-35B-heretic \
  --mlx-path ./Ornith-1.0-35B-heretic-mlx-8bit \
  -q --q-bits 8 --q-group-size 64

Two fixes were required (documented for reproducibility):

1. mlx-vlm converter patch. mlx-vlm 0.6.3's qwen3_5_moe converter only handles a pre-fused expert layout (experts.gate_up_proj); this checkpoint uses the standard per-expert HF layout (experts.N.{gate,up,down}_proj.weight, ×256 experts), so it fails with KeyError: …experts.gate_up_proj. Patch mlx_vlm/models/qwen3_5_moe/qwen3_5_moe.py → sanitize() to stack the per-expert weights (numerically, so experts.10 doesn't sort before experts.2) into switch_mlp.{gate,up,down}_proj.weight.
2. vision_config type (for LM Studio). The upstream config declares vision_config.model_type = "qwen3_5_moe_vision", which LM Studio's VisionModel rejects (Unsupported model type: qwen3_5_moe_vision). This repo ships the config with vision_config.model_type = "qwen3_5_moe", which loads with full vision + tool calling in both LM Studio and mlx-vlm.

License

MIT, inherited from the base model deepreinforce-ai/Ornith-1.0-35B.

Credits

• Base model: deepreinforce-ai/Ornith-1.0-35B (DeepReinforce AI)
• Heretic / abliteration: thanet-s/Ornith-1.0-35B-heretic (Heretic 1.4.0)
• MLX quantization: this repo

⚠️ Safety notice

This is a Heretic-abliterated (uncensored) checkpoint: refusal behavior has been removed. It will generate content the base model would decline, including material that may be harmful, unsafe, illegal in some jurisdictions, or offensive. It is intended for local, personal, research use only. Do not deploy it in client-facing, automated, or production workflows without your own safety layer. You are responsible for how you use it.