Ornith-1.0-35B-oQ4e-XL-mlx

leonsarmiento/Ornith-1.0-35B-oQ4e-XL-mlx

A hybrid MLX quantization that surgically combines layers from two quantization variants of Ornith-1.0-35B:

• Attention, embeddings, shared expert, lm_head from leonsarmiento/Ornith-1.0-35B-5bit-XL-mlx — BaseQuant_XL 8-bit (data-agnostic)
• Routed experts (switch_mlp) from Jundot/Ornith-1.0-35B-oQ4e — oQ4e 4-bit (data-aware)

The result is a quantization-level layer swap — not a weight-space merge (DARE-TIES, Task Arithmetic). The model architecture is identical; only the quantization precision and method per layer differs.

About XL Quantization

BaseQuant_XL is a fully data-agnostic, static quantization. No calibration dataset, no sensitivity analysis, no importance matrix. Precision is allocated purely by architectural role — routing-critical layers get higher precision, bulk expert parameters get lower precision. The result is a transparent, faithful capture of the source model.

Data-dependent calibration quantizations (iMatrix, AWQ, GPTQ, oQ, oQ4e, etc.) use a calibration set to guide bit allocation. This can produce a skewed representation of the model: domains well-represented in the calibration data (English, popular topics, public or leaked benchmarks) are preserved better, while underrepresented domains (non-English languages, niche use cases, your own data) are preserved worse. XL avoids this trade-off entirely — it generalizes honestly because it is never fit to any particular data distribution.

Why This Combination?

The two donor quantizations use fundamentally different approaches:

The hypothesis: XL's high-precision 8-bit attention provides the best token routing and context understanding, while oQ4e's data-aware 4-bit experts preserve the weight regions that matter most for the oQ calibration data. This combines the strengths of both approaches.

Important consideration: oQ/oQe quantization is data-aware — its quality depends on how well the oQ calibration dataset represents your use case. If your workload aligns with the calibration data, data-aware experts can outperform data-agnostic quantization at the same or lower bit depth. If it doesn't, the data-agnostic XL quantization may be more reliable. This hybrid gives you data-aware experts with data-agnostic attention — a middle ground.

Benchmark Results (n=50)

Key findings (n=50, wide confidence intervals):

• Coding synergy is real. HUMANEVAL jumps to 94.0% — +26 over XL alone and +10 over oQ4e alone. The combination of precise 8-bit attention with data-aware 4-bit experts appears to be mutually reinforcing for code generation.
• LIVECODEBENCH tempers the coding story. While HUMANEVAL soars, LIVECODEBENCH at 42.0% trails XL's 48.0% by −6. The oQ4e data-aware 4-bit experts may lose precision on competitive-programming problems that demand exact algorithmic reasoning. HUMANEVAL's function-completion format may be more forgiving than LIVECODEBENCH's full-problem format.
• Commonsense reasoning trades down. WINOGRANDE drops 4–6 points below both parents. The 4-bit data-aware experts may lose nuance on word-association tasks that XL's 5-bit experts handled better.
• MMLU and MBPP are near-wash. Slight edge to oQ4e on knowledge recall and basic Python.

⚠️ Benchmarks were run in instruct mode only (no reasoning/thinking). Enabling reasoning mode is expected to improve scores further, but was omitted to save time and energy. n=50 samples means wide confidence intervals (±~13% at 95% CI). The HUMANEVAL jump is too large to dismiss as noise, but MMLU/WINOGRANDE/MBPP differences may not be statistically significant.

Quantization Strategy

About the Base Model

Ornith-1.0-35B is a 35B-parameter MoE (Mixture of Experts) model fine-tuned from Qwen3.5-35B-A3B by DeepReinforce AI, using a self-improving RL training framework that jointly optimizes scaffold and solution rollouts for agentic coding tasks. Despite 35B total parameters, only ~3B are activated per token. It features 256 experts (8 active per token + 1 shared expert), hybrid full + linear (Gated DeltaNet) attention, and a vision encoder.

Use with mlx

pip install -U mlx-vlm

python -m mlx_vlm.generate --model leonsarmiento/Ornith-1.0-35B-oQ4e-XL-mlx --max-tokens 256 --temperature 1.0 --top-p 1.0 --prompt "Hello"

Recommended Inference Parameters

Note: Ornith-1.0-35B uses Temp 1.0 and Top_p 1.0 per the model's Terminal-Bench 2.1 benchmark recipe. This is a Qwen3.5-based model — preserve_thinking is not applicable.

Acknowledgments

• Jundot for the oQ4e data-aware quantization and the oQ framework
• DeepReinforce AI for the base Ornith-1.0-35B model