micro-qwen4exp

Qwen3.8 Flash Next (qwen4exp) — Micro Debug Model

A tiny, randomly initialized GGUF model set featuring the exact Qwen3.8 Flash Next (qwen4exp) architecture. It is designed for debugging and profiling the QSA (sparse attention), GDN, MTP, and vision paths of llama.cpp on any backend.

The file layout matches the official Unsloth split: trunk, detached MTP head, and mmproj.

⚠️ Important: Random Weights — NOT for Inference

All weights are random noise. The model will produce garbage text and must be used only for:

• Speed profiling with llama-bench (the operations that scale with context length match the real model; see below).
• Graph build and load-path coverage testing (llama-quantize / test-llama-archs).
• Debugging the graph layout, memory management, attention indexer, MTP draft graph, and mmproj.
• A/B token-identity checks during refactoring (e.g., greedy decoding on the same prompt should yield the exact same "garbage" output).
• CI pipelines that require the qwen4exp architecture without needing a multi-GB download.

Do not use this model for any quality evaluations. The MTP acceptance rate is meaningless with random weights; measure that metric on the real model instead.

Why This Model Exists

The full Qwen3.8 Flash Next model is roughly 109 GB (UD-Q4_K_XL), plus a 2.5 GB MTP sidecar and a 904 MB mmproj. Debugging long-context regressions (such as the QSA indexer, Flash Attention over the full KV cache, or MTP rollbacks) with the full model is impractical. This micro set runs in seconds and reproduces the exact same performance-vs-context degradation curve, because the specific QSA dimensions that drive computational costs are kept identical to the real model.

At this reduced size, the n_embd matrix multiplications drop out of the profiling results. What remains are the operations that scale with context depth—which is exactly the point.

Files

The files serve the same three roles as the official Hugging Face / Unsloth release:

Architecture vs. the Real Model

The dimensions that determine QSA cost and RoPE are copied 1:1 from Qwen3.8-Flash-Next-UD-Q4_K_XL:

The following parameters are scaled down (affecting only matmul FLOPs and file size):

The layer recipe is identical to the real model, just truncated after three repeats:

L00 GDN+MoE+HC
L01 GDN+MoE+HC+PLE     ← only PLE layer, same as real
L02 GDN+MoE+HC
L03 QSA+MoE+HC
L04–06 GDN  L07 QSA
L08–10 GDN  L11 QSA

The trunk uses compress_ratios = [0,0,0,4, 0,0,0,4, 0,0,0,4]. The MTP sidecar appends a trailing 0 (dense full-attention with indexer tensors), exactly like the real blk.48.

The QSA inner dimensions match the real model on the head axis:

• attn_q [n_embd, 12288] (24×256×2)
• attn_k / attn_v [n_embd, 512]
• attn_output [6144, n_embd]
• indexer q_proj [n_embd, 512], k_proj [n_embd, 128]

The sparse window is top_k + compress_ratio − 1 = 2051 cells, with a block size of 4—identical to the real model. Indexer weights remain in F16 even through Q4_K_M quantization (llama.cpp does not quantize them; the real UD-Q4_K_XL keeps them in BF16).

MTP (Detached Head)

This works purely as a graph and runtime path test, not as a quality test.

The sidecar is an mtp_only qwen4exp GGUF: block_count=13, nextn_predict_layers=1, and no trunk tensors. blk.12 holds QSA + MoE + HC + nextn.{enorm,hnorm,eh_proj}. llama.cpp detects it the same way it does the real Qwen3.8-Flash-Next-MTP-Q4_K_M.gguf (i.e., missing blk.0.hc_attn_norm but present blk.<n_layer>.nextn.eh_proj).

Enable it using --mtp and -md micro-qwen4exp-realqsa-mtp-Q4_K_M.gguf. Again, the acceptance rate on random weights is just noise; use the real model for quality metrics. The primary reason this exists is to provide coverage for speculative-decode rollbacks and conv-state snapshots.

Vision Adapter

micro-qwen4exp-realqsa-mmproj.gguf is a micro clip / qwen3vl_merger mmproj. It features 2 ViT layers, 256 dimensions, 8 heads, a patch size of 16, and a spatial merge of 2. The projector uses mm.0 → GELU → mm.2 into the 768-d LLM. (For comparison, the real mmproj has 27 layers at 1152-d / 16 heads / proj 2560). It uses the exact same tensor names and projector type, but it is not meant to be a functional quality encoder.

Usage: --mmproj micro-qwen4exp-realqsa-mmproj.gguf

Limitations (Not the Real Model)

• Uses a dummy 32k tokenizer, not the full 248,320-merge Qwen3.5 tokenizer.
• The PLE hash table is tiny (the mixers are real, but the gather rows are not).
• Contains no trained weights. Vision and MTP quality are effectively zero.
• The SSM, MoE, and n_embd sizes are smaller, so GDN-FLOP and expert-FLOP profiling are not representative. Only the context-scaling QSA/FA/indexer path is accurate.

Usage

# Speed profiling (main use case) — Q4_K_M fits in small VRAM
llama-bench -m micro-qwen4exp-realqsa-Q4_K_M.gguf -ngl 999 -fa 1 \
    -d 2048,8192,32768,65536

# MTP draft graph (acceptance rate is meaningless here)
llama-server -m micro-qwen4exp-realqsa-Q4_K_M.gguf \
    --mtp -md micro-qwen4exp-realqsa-mtp-Q4_K_M.gguf

# Vision adapter load
llama-mtmd-cli -m micro-qwen4exp-realqsa-Q4_K_M.gguf \
    --mmproj micro-qwen4exp-realqsa-mmproj.gguf

# Architecture correctness (uses a synthetic qwen4exp fixture, not these specific files)
test-llama-archs -a qwen4exp
test-backend-ops -b Vulkan0 -o TOPK_QSA

This works with any llama.cpp build from roughly b10600 and newer that supports qwen4exp and detached MTP.

How It Was Generated

The script probe/mk_micro_gguf.c writes the three F16 GGUF files directly. It uses the real QSA, M-RoPE, PLE mixers, and tensor names, but keeps everything else micro-sized and initializes the weights randomly. After that, they are quantized:

llama-quantize --max-buffer-size 256 \
    micro-qwen4exp-realqsa.gguf micro-qwen4exp-realqsa-Q4_K_M.gguf Q4_K_M
llama-quantize --max-buffer-size 256 \
    micro-qwen4exp-realqsa-mtp.gguf micro-qwen4exp-realqsa-mtp-Q4_K_M.gguf Q4_K_M

To rebuild the F16 set from scratch:

cc -O2 -o probe/mk_micro_realqsa probe/mk_micro_gguf.c \
   -I llama.cpp/ggml/include -L llama.cpp/build/bin -lggml-base \
   -Wl,-rpath,$PWD/llama.cpp/build/bin -lm
./probe/mk_micro_realqsa .