Ornith-1.0-35B-ThinkingCap

Ornith-1.0-35B — Thinking-Cap

This is a follow-up to my earlier fine tune of 9B Ornith with Thinking-Cap GRPO training. It's a LoRA fine-tune of deepreinforce-ai/Ornith-1.0-35B (a Mixture-of-Experts coding model, ~3B active params).

Reinforcement learning taught it to compress its reasoning where that's safe with ~24% fewer tokens on math at higher accuracy — while keeping load-bearing code reasoning intact. It's the same ThinkingCap method (correctness-gated, length-penalized GRPO) — originally from BottleCap AI — proven on the 9B, applied here to the larger MoE base.

This was a touch of a struggle - fine-tuning an MoE model is a different ball of wax from a dense model and takes more memory since the experts also need memory space. In this case, on my nVidia GB10 box, I didn't have enough RAM to train the experts, so this adapter just hits the attention stack.

Like the 9B adapter, I pretty much just pointed Claude at the problem and let it cook. Much of the tech specs here is based on Claude's writeup.

Fundamentally, the training is on the full attention layers, as well as the linear-attention/GDN layers. 10 full attention, and 30 Gated DeltaNet. What I found interesting is that due to the inability to train the experts' weights, this tune wasn't able to compress the reasoning traces as much, but it still improved accuracy somewhat.

TL;DR

• What: RL LoRA fine-tune that cuts reasoning-token spend while holding correctness.
• How much: −24% tokens on GSM8K at +6 pts accuracy; code reasoning kept. Total-token cut is a more modest ~8%.
• Accuracy: average improved (80.7% → 83.3%), zero code regression (HumanEval ties stock, MBPP improves).
• Behavior: 0% loop rate — calmer under uncertainty than the base, not degenerate.
• Base: Ornith-1.0-35B (MIT) → post-trained on Qwen3.5 (Apache 2.0). This model: MIT. See License.

Usage

This is a LoRA adapter (~130 MB), applied on top of the base MoE deepreinforce-ai/Ornith-1.0-35B (the base is large — plan for ~70 GB of bf16 weights, or use a quantized base). Want a ready-to-run quantized build instead? See the GGUF companion repo khudgins/Ornith-1.0-35B-ThinkingCap-GGUF (Q8_0 recommended).

Quick use — transformers + PEFT:

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer
from peft import PeftModel

BASE = "deepreinforce-ai/Ornith-1.0-35B"
ADAPTER = "khudgins/Ornith-1.0-35B-ThinkingCap"   # this repo

tok = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForImageTextToText.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER).eval()

msgs = [{"role": "user", "content": "How many positive integers under 1000 are divisible by neither 5 nor 7?"}]
inp = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt",
                              return_dict=True).to(model.device)
out = model.generate(**inp, max_new_tokens=1024, do_sample=False)
print(tok.decode(out[0][inp["input_ids"].shape[1]:], skip_special_tokens=True))

Hosting — vLLM (LoRA served on the base):

vllm serve deepreinforce-ai/Ornith-1.0-35B \
  --enable-lora \
  --lora-modules thinking-cap=khudgins/Ornith-1.0-35B-ThinkingCap \
  --max-lora-rank 32
# then call the OpenAI-compatible API with  "model": "thinking-cap"

Results

Greedy decode, in-process (transformers + PEFT), N=50 per benchmark. Each cell is accuracy @ mean completion tokens.

Adaptive concision. The cut scales to how compressible each task is — dramatic on math (reasoning is cheap), near-zero on code (reasoning is load-bearing and the model keeps it). Total token reduction reads modest (~8%) precisely because the code benchmarks dominate the token count and code is deliberately not compressed. Accuracy holds or improves on every benchmark; the checkpoint sweep plateaus and later checkpoints begin to over-compress GSM8K (ckpt-800 drops to 88%), so ckpt-500 — the knee — is the released finalist, not the final step.

Does it loop? No.

Tested on 24 loop-prone prompts (hard reasoning, eval-framing, ambiguous), greedy decode, with a repeated-span detector:

No loops; the tune reduces runaway/truncation on adversarial prompts.

Method

Correctness-gated, length-penalized GRPO (RL with verifiable rewards). Per completion:

reward = correctness                       # 1.0 if answer/tests pass, else 0.0
       − (λ · normalized_length  if correct)   # length penalty ONLY when correct
       + format_bonus

The gate — length penalty applies only to correct answers — makes it impossible to trade accuracy for brevity. GRPO's group-relative advantage means compression happens only on prompts already mastered; a KL leash to the frozen base and a λ ramp keep it stable and self-limiting.

MoE specifics (how this differs from the dense 9B). The base is a 256-expert MoE. LoRA targets all attention layers (both full-attention and linear-attention/GDN projections, ~32.5M trainable params); the experts are frozen. This is partly structural: on this architecture the routed experts are stored as fused parameter tensors (a grouped GEMM), not standard linear layers, so a stock PEFT LoRA cannot attach to them — it adapts attention and the shared/dense MLP only. Freezing the experts also preserves the base model's knowledge and improves MoE-RL stability. One consequence: because "how much to elaborate" is substantially an FFN/expert behavior, steering through attention alone is a narrower channel than the dense 9B (which did adapt its MLP) — a likely contributor to the 35B's milder compression. Trained with plain transformers + PEFT + TRL GRPOTrainer (LoRA r32/α64) on an NVIDIA DGX Spark (GB10). Reward reuses the same verifiable checkers used to grade the model, so training and eval share one oracle.

Intended use & limitations

• Use: reasoning and coding assistant where you want the base's accuracy without paying for verbose chain-of-thought — the compression is strongest on math-style reasoning, and code responses stay appropriately detailed.
• Limitations: English-focused; inherits base Ornith/Qwen3.5-MoE limitations. Evaluation N is 50 per benchmark (smaller than the 9B's full-N run), so treat single-point differences as indicative, not definitive. As with any model, verify correctness-critical output.

License

MIT. This fine-tune — the LoRA adapter and any weights merged from it — is released under the MIT License. It adds no new base weights; it's a low-rank delta trained on top of the base.

Upstream provenance and terms, stated precisely:

• Base model: deepreinforce-ai/Ornith-1.0-35B is tagged MIT on Hugging Face; the inherited base weights are governed by that grant.
• Underlying architecture: Ornith-1.0-35B is post-trained on Qwen3.5, released under Apache 2.0. Apache 2.0's terms (notably its patent grant and NOTICE requirements) apply to the Qwen3.5-derived weights inherited through the base.
• Not a Gemma derivative. The Ornith family card also lists Gemma-4-based variants, which would carry Google's Gemma license. This model is not one of those — it derives solely from the Qwen3.5-based, MIT-tagged Ornith-1.0-35B (verified: Qwen3_5MoeForConditionalGeneration).

Reproduction & provenance

Method: ThinkingCap (correctness-gated, length-penalized GRPO) — an independent reproduction of the ThinkingCap model series by BottleCap AI (announcement · bottlecapai/ThinkingCap-Qwen3.6-27B), applied here to a larger Mixture-of-Experts coding model. Not affiliated with or endorsed by BottleCap AI.

Base: deepreinforce-ai/Ornith-1.0-35B (MIT per its HF tag) → post-trained on Qwen3.5 (Apache 2.0). Tooling: transformers, PEFT, TRL GRPOTrainer. Full recipe and eval scripts: github.com/khudgins/ornith-thinking-cap.