Qwen3.8-32B-Jumbo

Qwen3.8-32B-Jumbo (bf16)

A 76-layer, 31.9B-parameter model created by transplanting chunks of layers from Qwen3.6-27B into Qwen3.8-27B.

Jumbo is the inverse of pruning: instead of removing layers to make a model smaller, layers are added from a sibling model to make it larger and potentially restore knowledge that was lost during continued training between versions 3.6 and 3.8.

Qwen3.8-27B and Qwen3.6-27B share the same architecture and were confirmed via alignment census to share a weight basis (global median cosine similarity 0.917). Qwen3.8 is a continued-train of Qwen3.6 that traded some breadth of knowledge for improved code and agentic capabilities.

Jumbo restores that breadth by transplanting the three most-changed organs from Qwen3.6 back into Qwen3.8, adding them as additional layers rather than replacing the originals.

Chunk Selection Methodology

Alignment Census

Layer chunk alignment census between Qwen3.8 (stock) and Qwen3.6 (donor), based on computing the cosine similarity of every weight tensor across all 64 layers grouped into 16 chunks (4 layers each).

Selected chunks 3, 5, and 10:

• Chunk 3 (cos 0.884): the single most-drifted organ, mid-stack MLP tissue where factual associations are concentrated
• Chunk 5 (cos 0.888): the second most-drifted organ, also mid-stack MLP
• Chunk 10 (cos 0.904): the lone outlier in the upper half of the network, providing coverage beyond the dense mid-stack cluster

This selection targets the tissue where 3.8 diverged most from 3.6, on the hypothesis that these are the sites where breadth of knowledge was overwritten for code/agentic capability.

Each selected donor chunk from Qwen3.6 is inserted immediately before its corresponding Qwen3.8 original at native depth. The donor chunk reads the same input stream it saw in Qwen3.6; the stock original sits downstream and absorbs the combined signal.

This produces a 76-layer model (64 original + 12 transplanted):

• Layers 0-11: stock chunks 0-2
• Layer 12-15: donor chunk 3 (from Qwen3.6)
• Layers 16-19: stock chunks 3 (original)
• Layers 20-27: stock chunks 4-5 (pre-transplant)
• Layers 28-31: donor chunk 5 (from Qwen3.6)
• Layers 32-35: stock chunk 5 (original)
• Layers 36-51: stock chunks 6-9
• Layers 52-55: donor chunk 10 (from Qwen3.6)
• Layers 56-59: stock chunk 10 (original)
• Layers 60-75: stock chunks 11-15

The embedding, final norm, and lm_head come from the stock Qwen3.8.

Key Properties

• Parameters: ~31.9B (vs 27.4B for the parent Qwen3.8)
• Layers: 76 (vs 64)
• Architecture: Qwen3 (identical hidden dim, heads, vocab)
• Knowledge: Restores factual breadth from Qwen3.6 while retaining 3.8's code and agentic capabilities
• MTP Compatible: Accepts the parent Qwen3.8 MTP drafter with ~82-89% acceptance rate, enabling speculative decoding speedups of 1.6-2.2x
• License: Apache-2.0 (inherited from both parents)

Quantized Versions

• Qwen3.8-32B-Jumbo-8bit
• Qwen3.8-32B-Jumbo-4bit
• Qwen3.8-32B-Jumbo-3bit

Usage (MLX)

# Standard generation
mlx_vlm.generate \
  --model StargazerLabs/Qwen3.8-32B-Jumbo \
  --prompt "Your prompt here" --max-tokens 2048

# With MTP speculative decoding (~1.6-2.2x speedup)
mlx_vlm.generate \
  --model StargazerLabs/Qwen3.8-32B-Jumbo \
  --draft-model mlx-community/Qwen3.8-27B-MTP-bf16 \
  --prompt "Your prompt here" --max-tokens 2048

Related Models

• Parent (stock): Qwen/Qwen3.8-27B
• Donor: Qwen/Qwen3.6-27B
• Mini-Me family (52-layer sheared models): StargazerLabs Mini-Me Collection

Part of the Jumbo Collection

StargazerLabs Jumbo Collection