Ornith-1.5-35B-A3B-BigBang-MTP

Ornith-1.5-35B-A3B-BigBang-MTP

A TIES merge of ornith-ai/Ornith-1.5-35B-A3B (agentic-coding RL) and endless-frontier/BigBang-v1 (general tune), both post-trains of Qwen/Qwen3.6-35B-A3B — plus a working MTP speculative-decoding head, which stock Ornith-1.5 does not have.

The MTP finding

Ornith-1.5-35B-A3B ships 785 mtp.* tensors that are random initialisation, not trained weights: every projection has std = 0.0200 with Gaussian kurtosis 3.0 (i.e. exactly initializer_range=0.02), and its norm weights sit near 0.02 instead of ~1. Used as a speculative draft it accepts only ~13% of tokens (pure chance). This model replaces that placeholder with the trained MTP head from Qwen3.6-35B-A3B, which transfers cleanly because Ornith's language tower is only ~0.2–1% away from Qwen3.6 (measured cosine per tensor group).

Measured with llama.cpp speculative decoding (Q4_K_M main + Q8_0 draft, RTX 3090):

Merge recipe

• TIES (density 0.25, λ=1.0), computed in fp32 over the Qwen3.6-35B-A3B base, on all text weights.
• MoE router gates: kept verbatim from Ornith (never averaged — routing is where naive MoE merges break).
• Vision tower: kept verbatim (bit-identical between Ornith and Qwen3.6 anyway; this model keeps Qwen3.6's multimodal eyes).
• mtp.*: Qwen3.6's trained head, verbatim (fused-expert layout, loads with the same Qwen3_5MoeForConditionalGeneration class).

Evaluation (Q4_K_M, single RTX 3090, temp 0.1)

Chat, instruction-following, translation and creative prompts remain coherent (spot-checked; e.g. Welsh translation is understandable but slightly unnatural). This is a small local eval, not a benchmark suite — treat it as a no-regression check plus the MTP head-to-head, not a leaderboard claim.

Notes and caveats

• Reasoning is always on (<think>), inherited from Ornith/Qwen3.6. Give generous max_tokens.
• Delta-interference probes (cosine between task vectors) showed Ornith's RL delta and BigBang's tune are near-orthogonal (cos ≈ +0.11), which is why the merge composes; this does not guarantee gains on any specific benchmark.
• GGUF quants + the MTP draft GGUF: EryriLabs/Ornith-1.5-35B-A3B-BigBang-MTP-GGUF
• Licences: Ornith-1.5 is MIT; BigBang-v1 and Qwen3.6-35B-A3B are Apache-2.0. This merge is released under MIT with attribution to all three parents.

Merged and measured by EryriLabs (Dwain Barnes), 2026-08-20.