Ornith-1.5-9B-vision-OptiQ-static-4.5bpw

Ornith-1.5-9B-vision-OptiQ-static-4.5bpw

Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon. Docs · All OptiQ quants

A static mixed-precision MLX quant of ornith-ai/Ornith-1.5-9B, the dense 9B Ornith 1.5 vision-language model (Qwen3.5, 32 layers). Produced with optiq convert --method static --target-bpw 4.5 --candidate-bits 4,5,6,8 --reference auto.

This checkpoint takes images and text. The language tower is mixed-precision affine MLX. The vision tower is kept at bf16 in optiq/optiq_vision.safetensors. Stock mlx-lm loads the language tower only. Image input needs mlx-optiq.

Sizes below are decimal GB, matching what the Hub sidebar reports.

Source

This is a static convert, not a KL-measured OptiQ allocation, and there is no uniform 4-bit grid in the file. Bit-widths come from architecture rules, not from a sensitivity pass on Ornith 1.5.

Quantization details

The name uses 4.5bpw (the convert target), not "4-bit". --candidate-bits 4,5,6,8 is why the language mix includes 5-bit and a 6-bit leftover, not only 4 and 8. The bf16 vision tower is unquantized.

Reproduce:

pip install "mlx-optiq>=0.4.27"
optiq convert ornith-ai/Ornith-1.5-9B --target-bpw 4.5 --candidate-bits 4,5,6,8 --reference auto --method static -o ./optiq_output/Ornith-1.5-9B-vision-OptiQ-static-4.5bpw

Image input

The vision tower is not quantized. 4-bit vision tends to hurt OCR and fine detail, so OptiQ leaves it at bf16. Processor configs (preprocessor_config.json, processor_config.json, video_preprocessor_config.json) are copied from the base so the Hub artifact is a complete image-text-to-text package.

Image path requires mlx-optiq 0.4.27 or newer (sidecars live under optiq/, not the repo root).

pip install "mlx-optiq>=0.4.27"

from PIL import Image
from optiq.runtime.engine import OptiqEngine

engine = OptiqEngine("ahmedihamdy/Ornith-1.5-9B-vision-OptiQ-static-4.5bpw")
answer = engine.generate(
    "What is in this image?",
    images=[Image.open("photo.jpg")],
    max_tokens=512,
)
print(answer.text)

OpenAI- and Anthropic-compatible serving with vision:

optiq serve --model ahmedihamdy/Ornith-1.5-9B-vision-OptiQ-static-4.5bpw

Text

A stock *.safetensors glob ignores optiq/, so mlx-lm sees a clean language model. Register the arch and sidecars first:

pip install mlx-lm "mlx-optiq>=0.4.27"

import optiq  # registers the arch + MTP/vision sidecars
from mlx_lm import load, generate

model, tokenizer = load("ahmedihamdy/Ornith-1.5-9B-vision-OptiQ-static-4.5bpw")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain the difference between TCP and UDP."}],
    add_generation_prompt=True,
    tokenize=False,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))

This is a reasoning model: it thinks before answering, so give max_tokens room to finish.

Speculative decoding (MTP)

This quant bundles a Multi-Token Prediction head at optiq/mtp.safetensors.

optiq serve --model ahmedihamdy/Ornith-1.5-9B-vision-OptiQ-static-4.5bpw --mtp

What this is not

• Not a KL-measured OptiQ allocation on Ornith 1.5.
• Not a uniform 4-bit quant. Language tensors are a 4/5/6/8-bit mix.
• Not a quantized vision tower. mlx-lm text-only loads will not see images.
• No public MMLU / GSM8K / SWE-bench numbers for this file.

Quantization does not change the behaviour or alignment of the base model. Use it under the same terms as ornith-ai/Ornith-1.5-9B.