ornith-1.0-9b-mxfp8-mlx

ornith-1.0-9b-mxfp8-mlx

MLX quantization of deepreinforce-ai/Ornith-1.0-9B for Apple Silicon.

Variant: Block float MX FP8
Disk size: 8826 MB
Quantized by: sahilchachra

About this model

Ornith-1.0-9B is an agentic coding & reasoning model from the DeepReinforce team, post-trained on top of Qwen 3.5 (9B-Dense). It is a reasoning model: each assistant turn opens with a <think> … </think> block before the final answer. Recommended sampling: temperature=0.6, top_p=0.95, top_k=20.

Text tower only. Ornith-1.0-9B is distributed in a Qwen3.5-VL multimodal container (Qwen3_5ForConditionalGeneration), but its trained capabilities are text / code. This MLX conversion keeps the language tower only — the vision encoder is dropped at conversion time by mlx-lm's qwen3_5 loader. Use it as a text/code LLM (it has no image input).

A note on the benchmark

Ornith's headline benchmarks (Terminal-Bench 2.1, SWE-bench Verified/Pro/Multilingual, NL2Repo, ClawEval, SWE Atlas) are agentic harnesses — they need Docker, a large GPU, and long multi-turn rollouts at up to 256K context, so they are not reproducible on a Mac. As an on-device check that quantization preserves coding ability, this card reports HumanEval pass@1 (30 problems, 4096-token budget, with reasoning enabled and <think> stripped before grading). Both MX FP4 and MX FP8 score 93.3% (28/30) with 100% syntactic validity — i.e. 4-bit quantization is effectively lossless on this proxy. HumanEval is a narrow code-generation probe and is not a substitute for the agentic evaluations on the base model card.

This is the reference variant. MX FP8 is near-lossless and is used as the quality/performance reference for the other variants in this collection, so this card reports its absolute numbers (no comparison column).

Benchmark results

Evaluated on Apple M5 Pro with MLX. Model loaded once; performance and quality measured in a single pass.

Performance

Quality

Context scaling (decode tok/s)

Usage

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("sahilchachra/ornith-1.0-9b-mxfp8-mlx")
response = generate(model, tokenizer, prompt="Your prompt here", max_tokens=256, verbose=True)

All variants in this collection

Notes

• Requires Apple Silicon (M1 or later) with MLX
• Benchmarks run on Apple M5 Pro, 24 GB unified memory
• License: see deepreinforce-ai/Ornith-1.0-9B for the original model's license

Original model

See deepreinforce-ai/Ornith-1.0-9B for full model details and intended use.