Ornith-1.0-9B-MLX-4bit

Ornith-1.0-9B-MLX-4bit

Summary

MLX 4bit (affine, 4-bit, group size 64) quantization of deepreinforce-ai/Ornith-1.0-9B at upstream revision 83dc1f5e24ef8527af019a6b3bf66ac0f1c2c999, converted from the BF16 safetensors with mlx_vlm convert (mlx-vlm 0.6.3). ~5.6 GiB on disk.

Runtime status (verified 2026-07-04): loads and generates under mlx-vlm 0.6.3 (vision probe passed on this exact payload) and under mlx-lm 0.31.3 for text-only use (text probe passed). Republished 2026-07-04 in the mlx-vlm module layout, replacing the 2026-07-02 pack, which loaded text-only under mlx-lm and not at all under mlx-vlm. Output quality beyond the smoke probes is not verified.

Runtime status

Smoke-gated on Apple Silicon before publish (2026-07-04): this exact payload was loaded and probed per variant (pipelines/vlm_pack_smoke.py / pipelines/lm_pack_smoke.py, majek repo); verdicts in .sisyphus/evidence/lane-a-republish/ and in PROVENANCE.md.

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("majentik/Ornith-1.0-9B-MLX-4bit")
prompt = apply_chat_template(
    processor, model.config, "Describe this image.", num_images=1
)
print(generate(model, processor, prompt, image=["image.jpg"],
               max_tokens=128).text)

Text-only use:

from mlx_lm import load, generate

model, tokenizer = load("majentik/Ornith-1.0-9B-MLX-4bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain KV caching in one sentence."}],
    add_generation_prompt=True, tokenize=False,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=128))

Why this variant

Affine 4-bit, group size 64 — the standard MLX deployment tier and the usual quality/size sweet spot.

Reproduce

# base = snapshot of deepreinforce-ai/Ornith-1.0-9B @ 83dc1f5e24ef8527af019a6b3bf66ac0f1c2c999
python -m mlx_vlm convert --hf-path /tmp/lane-a-republish/ornith-1.0-9b/base --mlx-path /tmp/lane-a-republish/ornith-1.0-9b/4bit -q --q-bits 4 --q-group-size 64 --q-mode affine

Vision tower

The vision tower (333 upstream tensors) rides through in BF16 — mlx_vlm convert quantizes only the language tower by default. This pack stores weights in the mlx-vlm module layout (vision_tower.*, language_model.*) rather than the upstream model.visual.* naming.

Family

All MLX variants of this model ship together:

• majentik/Ornith-1.0-9B-MLX-4bit (this repo)
• majentik/Ornith-1.0-9B-MLX-6bit
• majentik/Ornith-1.0-9B-MLX-8bit
• majentik/Ornith-1.0-9B-MLX-MXFP4

Provenance

• Upstream: deepreinforce-ai/Ornith-1.0-9B @ 83dc1f5e24ef8527af019a6b3bf66ac0f1c2c999
• Quantization: bits=4, mode=affine, group_size=64 (language tower only)
• Toolchain: mlx 0.31.2, mlx-lm 0.31.3, mlx-vlm 0.6.3, transformers 5.12.0, huggingface_hub 1.22.0
• Republished 2026-07-04 in a runtime-loadable layout; supersedes the 2026-07-02 pipelines.mlx_direct_quantize pack. Full details in PROVENANCE.md in this repo.

License + attribution

Quantized by majentik from deepreinforce-ai/Ornith-1.0-9B. All rights in the original model remain with its authors.

Upstream declares the MIT license via its model-card metadata but ships no LICENSE file, so the full MIT license text is reproduced below with attribution to deepreinforce-ai:

MIT License

Copyright (c) deepreinforce-ai

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.