Downloads · 30 days
112
13% of all-time downloads
RukaRat/Ornith-1.5-35B-A3B-oQ6-gs128-mtp-vision
Ornith-1.5-35B-A3B-oQ6-gs128-mtp-vision is a image-text-to-text model from RukaRat. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as mit.
A 6-bit oQ quantization of shisa-ai/Ornith-1.5-35B-A3B-MTP for oMLX on Apple Silicon, keeping both the vision tower and a functioning MTP head, and quantized at groupsize 128 rather than the usual 64.
Downloads · 30 days
112
13% of all-time downloads
All-time downloads
896
Public
Parameters
36B
30.1 GB on disk
Likes
2
Public
Click a slice to open those files.
.safetensors30 GB · 100%
How the weights are stored.
U3235.5B · 99%
From the Hugging Face model README
A 6-bit oQ quantization of shisa-ai/Ornith-1.5-35B-A3B-MTP for oMLX on Apple Silicon, keeping both the vision tower and a functioning MTP head, and quantized at group_size 128 rather than the usual 64.
vision_config + 333 vision tensors + preprocessor_config.json)Two independent problems with the common Ornith MLX quants:
1. Stock Ornith's MTP head does nothing. Measured on an M2 Max under oMLX 0.6.2:
| raw (MTP off) | MTP on | gain | |
|---|---|---|---|
| Qwen3.6-35B-A3B oQ6 | 72.50 tok/s | 80.11 | +10.5% |
| Ornith-1.5 oQ5 (stock head) | 72.68 tok/s | 72.81 | +0.2% |
Raw autoregressive speed is identical; stock Ornith's MTP verify forward costs almost exactly the 1.77× it produces, so speculative decoding is break-even. Shisa AI's source fixes this by replacing the native head with a Qwen3.6-initialized one trained by full-vocabulary KL distillation. This build starts from that fixed source.
Most published Ornith oQ builds list base_model: ornith-ai/Ornith-1.5-35B-A3B — i.e. the
stock head — so their MTP is largely inert.
2. group_size=64 costs ~9%. Against
pyros-vault/Ornith-1.5-35B-A3B-oQ6e-fixed-mtp,
which uses the same fixed head and is otherwise identical (6-bit affine, vision, 42 MTP
tensors, 6.89 vs 6.87 bpw) but group_size=64:
6 paired A/B rounds, server restarted before every measurement
per-pair delta: [+6.0, +11.5, +7.6, +9.9, +6.5, +10.9] %
mean +8.7% stdev 2.1% this build ahead 6/6
MTP acceptance was the same in both (78–84%, ~2.1 tok/cycle), so this is not better
drafting. At equal bits-per-weight, group_size=64 creates twice as many scale/bias arrays
and the dequantization path pays double the per-group overhead. The cost is kernel overhead,
not memory bandwidth — which is why reducing bpw (oQ5, target_bpw) did not help.
MLX supports group sizes 32/64/128 only, so 128 is the maximum.
from omlx.oq import quantize_oq_streaming
quantize_oq_streaming(
model_path="shisa-ai/Ornith-1.5-35B-A3B-MTP",
output_path="Ornith-1.5-fixedmtp-oQ6-lean",
oq_level=6, group_size=128, # 128, not the default 64
dtype="float16", # M2/M1 have no native bf16
preserve_mtp=True, text_only=False, # keep the MTP head and the vision tower
target_bpw=6.5, hard_cap_bpw=6.7,
auto_proxy_sensitivity=True, # required: 70GB exceeds live calibration memory
)
MTP must be enabled explicitly — it is off by default and fails silently:
// ~/.omlx/model_settings.json
{"models": {"Ornith-1.5-fixedmtp-oQ6-lean": {"mtp_enabled": true}}}
Verify it actually engaged with grep "MTP\[" ~/.omlx/logs/server.log — look for
accept= and tok/cycle=. Absence means it is not running.
Note mtp_enabled and vlm_mtp_enabled are mutually exclusive; setting both makes the
whole settings file fail validation and silently revert to defaults.
Tested against Jundot/Qwen3.6-35B-A3B-oQ6-fp16-mtp on the same machine:
days_ago int, extensions array), this build was
6/6; Qwen3.6 emitted no tool call on 4 of 6 runs (2/6).finish=stop on a one-sentence constraint).