GrugCap-27B

GrugCap-27B

A layer splice, not a finetune. Decoder blocks 10–44 of ProCreations/grug-27b were copied byte-for-byte into bottlecapai/ThinkingCap-Qwen3.6-27B, replacing that range wholesale. Everything outside the range is ThinkingCap's.

Both parents are Apache-2.0 finetunes of Qwen/Qwen3.6-27B, which is what makes a raw weight-space splice coherent: their residual streams share a basis, so transplanted layers land in a space the surrounding layers still understand. Splicing across unrelated pretraining runs does not work this way and produces noise.

The premise being tested is that the middle depth band of a transformer carries most of the feature-composition work, so swapping it swaps "how the model thinks" while the outer layers keep the host's tokenization behaviour, output formatting and answer style. Blocks 10–44 are 15–70% of depth, a heuristic choice, not a discovered boundary.

Composition

463 of 1199 tensors come from grug: 24.8 GiB of 51.7 GiB, or 47.9% by weight. The 35 spliced blocks are 26 linear-attention and 9 full-attention layers, the hybrid pattern placing full attention every fourth layer.

Provenance was verified after the build by hashing tensors in the output against the same tensors fetched from both parent repos. The boundary is exact: blk.44.mlp.down_proj matches grug and not ThinkingCap, blk.45.mlp.down_proj matches ThinkingCap and not grug, with no unexplained tensors.

A useful incidental finding: the two parents have byte-identical embed_tokens, lm_head, every layernorm and every conv1d weight. They differ only in projection matrices, in all 64 blocks. That is the expected signature of grug's LoRA-on-linears method, and it means a provenance check that samples only layernorms cannot distinguish the parents at all.

What was tested

On a Q8_0 GGUF under llama.cpp on 2×RTX 3090:

• Loads and generates cleanly; thinking mode intact and correctly delimited.
• A small battery of reasoning prompts with deterministic answers: 4/4 on the auto-graded items, with the free-form items (bracket-matching code, strict JSON formatting, a 24-game expression, a deliberately unsatisfiable ordering puzzle) all correct on inspection.
• 220.6 tok/s prompt processing, 25.2 tok/s generation.

Thinking traces are terse and enumerative, closer to grug's register than to a verbose reasoner. Mean trace length across the battery was 155 characters. On the 24-game prompt it enumerated ten dead ends and verified the winner in 438 characters.

What was NOT tested

Be skeptical of this model until someone runs real evals. In particular:

• No head-to-head comparison against either parent. No claim is made that this is better, or even as good.
• No standard benchmarks (GSM8K, MMLU-Pro, GPQA, HumanEval). The battery above is a smoke test, not an evaluation.
• No perplexity measurement.
• No long-context, multi-turn, agentic, or tool-use testing.
• Vision inputs untested, though the tower and projector are carried through.
• No check for seam artifacts at the two splice boundaries, where a short healing finetune would be the obvious remedy if quality problems appear.

Files

BF16 safetensors in 11 shards, plus model-base-aux.safetensors holding ThinkingCap's MTP draft head. That aux file is not in the weight index; move it out of the directory before running convert_hf_to_gguf.py or the converter will trip over it.

Reproducing

python splice_remote.py \
  --recipient bottlecapai/ThinkingCap-Qwen3.6-27B \
  --donor ProCreations/grug-27b \
  --start 10 --end 45 \
  --out ./GrugCap-27B

The splice is IO-bound, so re-cutting at different boundaries is cheap relative to any training. If you want a different band, change --start/--end; the tooling streams tensors by HTTP range request and never stores either parent locally.

Credits

• bottlecapai/ThinkingCap-Qwen3.6-27B — host model, Apache-2.0
• ProCreations/grug-27b — donor blocks, Apache-2.0
• Qwen/Qwen3.6-27B — shared base

Neither parent's authors were involved in or endorse this merge.