Ornith-Agents-A1-3.6-35B-A3B-task_arithmetic_v2-6bit-XL-mlx

leonsarmiento/Ornith-Agents-A1-3.6-35B-A3B-task_arithmetic_v2-6bit-XL-mlx

This model was converted to MLX format from tepirale/Ornith-Agents-A1-3.6-35B-A3B-task_arithmetic_v2 using BaseQuant_XL 6/8-bit mixed quantization optimized for Apple Silicon. The vision encoder is preserved and quantized at 6-bit, making this a full multimodal model.

BaseQuant_XL keeps the most routing-critical layers in full bf16 precision — the MoE router gate, shared expert gate, shared expert, and lm_head — while applying aggressive quantization to the bulk parameters. This preserves routing accuracy and output quality where it matters most.

About XL Quantization

BaseQuant_XL is a fully data-agnostic, static quantization. No calibration dataset, no sensitivity analysis, no importance matrix. Precision is allocated purely by architectural role — routing-critical layers get higher precision, bulk expert parameters get lower precision. The result is a transparent, faithful capture of the source model.

Data-dependent calibration quantizations (iMatrix, AWQ, GPTQ, oQ, oQ4e, etc.) use a calibration set to guide bit allocation. This can produce a skewed representation of the model: domains well-represented in the calibration data (English, popular topics, public or leaked benchmarks) are preserved better, while underrepresented domains (non-English languages, niche use cases, your own data) are preserved worse. XL avoids this trade-off entirely — it generalizes honestly because it is never fit to any particular data distribution.

Model Description

This is a Task Arithmetic merge of two complementary Qwen3.5-35B-A3B agentic models onto the base model:

Merge configuration:

merge_method: task_arithmetic
base_model: Qwen/Qwen3.5-35B-A3B
dtype: bfloat16
parameters:
  normalize: true
models:
  - model: Qwen/Qwen3.5-35B-A3B
    parameters:
      weight: 0.3
  - model: InternScience/Agents-A1
    parameters:
      weight: 0.6
  - model: deepreinforce-ai/Ornith-1.0-35B
    parameters:
      weight: 0.6

Both fine-tuned models receive equal weight (0.6) as task vectors applied to the base, with the base retaining a lower weight (0.3). normalize: true ensures task vectors are normalized. This emphasizes the task-specific capabilities of both fine-tuned models while maintaining stability from the base.

The architecture is a 35B Mixture-of-Experts model with only ~3B active parameters per token, featuring 256 experts (8 active + 1 shared), hybrid full + linear (Gated DeltaNet) attention, a vision encoder, and an extended 262K context window.

Sibling model: A DARE-TIES merge of the same parent models is also available — compare the two merge methods in the Ornith-Agents-A1 Merge collection.

Use with mlx

pip install -U mlx-vlm

python -m mlx_vlm.generate --model leonsarmiento/Ornith-Agents-A1-3.6-35B-A3B-task_arithmetic_v2-6bit-XL-mlx --max-tokens 256 --temperature 0.85 --top-p 0.95 --top-k 20 --min-p 0.01 --repeat-penalty 1.05 --prompt "Hello"

BaseQuant_XL Quantization Strategy

Quantization Details

• Quantization type: BaseQuant_XL mixed (multimodal, vision preserved)
• Group size: 64
• Method: Custom quant_predicate via mlx_vlm

Recommended Inference Parameters

Inherited from Agents-A1 (more conservative, broader tool-use optimization):

Note: These use Agents-A1's recommended settings rather than averaging across both parents. Ornith's original values (temp 1.0, top_p 1.0, top_k 40) were optimized for Terminal-Bench specifically — Agents-A1's broader agentic profile is a safer default for general use. Adjust as needed.

Benchmarks (n=30, 5-bit XL MLX)

The Task Arithmetic merge achieves the highest MMLU score (73.3%) across all models, inherits Agents-A1's coding strength (HUMANEVAL 86.7%), and matches parents on MATHQA (96.7%). Minor regressions on MMLU_PRO and TRUTHFULQA vs parents — typical task arithmetic trade-off. Both merges clearly dominate the Qwen3.6 base on knowledge and coding benchmarks.

Reasoning and Tool-Call Parsing

This is a Qwen3.5-based model — preserve_thinking is not applicable.