qwen3-14b-32b-kv-mapper-bf16-tp1-b1

Qwen3-14B to Qwen3-32B KV mapper, bf16 TP=1

This is the k=8 full-head ridge KV mapper artifact for qwen3-14b-32b-full_head-bf16-tp1-h8-s40c069824f4251a91eefaf281ebe4c544efd3e18-t9216db5781bf21249d130ec9da846c4624c16137-b1. It maps pre-RoPE source keys and source values to target KV tensors for an offline Qwen3-14B to Qwen3-32B cache-injection run.

Files

• manifest.json, weights.safetensors, and SHA256SUMS are the loadable artifact. Verify with sha256sum -c SHA256SUMS.
• run.json records the full calibration recipe and token-window fingerprint.
• evaluation/report.json gives paired bootstrap intervals.
• evaluation/items.json gives identified per-example scores for all arms.

Fit recipe

From the linked repository root, the collection and fit commands are:

python src/training/kv_mapper/collect_run.py \
  --source-model Qwen/Qwen3-14B \
  --source-revision 40c069824f4251a91eefaf281ebe4c544efd3e18 \
  --target-model Qwen/Qwen3-32B \
  --target-revision 9216db5781bf21249d130ec9da846c4624c16137 \
  --dataset-revision 87f09149ef4734204d70ed1d046ddc9ca3f2b8f9 \
  --dtype bf16 --num-kv-heads 8 --head-dim 128 --seed 42 \
  --num-sequences 500 --seq-len 1024 --window-stride 0 \
  --fitting-token-step 4 --output-dir ./calibration
python src/training/kv_mapper/fit_run.py \
  --run-dir ./calibration --output-dir ./artifacts \
  --pair-slug qwen3-14b-32b --topk 8 --ridge-alpha 0.01 \
  --bundle-version 1

Held-out offline evaluation

HellaSwag validation (Rowan/hellaswag revision 218ec52e09a7e7462a5400043bb9a69a41d06b76), 100 seeded sampled rows with stable row IDs. Each ending was scored with the same target model under cold target, mapped source, raw source, and zero-cache arms. The continuation scorer uses the same context and ending token IDs across arms.

Accuracy ranks choices by total ending log probability. Length-normalized accuracy ranks by mean log probability per ending token. The earlier 77% cold / 80% mapped figures were length-normalized accuracy; the corrected report gives both metrics explicitly.

Paired mapped-minus-cold accuracy: -0.03 (95% bootstrap interval [-0.07, +0.01]). Paired length-normalized accuracy: +0.03 (95% interval [-0.01, +0.07]). Paired mean gold-ending log probability per token: +0.0123 (95% interval [-0.0328, +0.0637]).

Mean pre-RoPE relative error across held-out items and layers was 0.3267 for mapped keys and 0.6026 for mapped values. The corresponding raw source errors were 1.3679 and 1.5992.

These results cover offline teacher-forced cache injection. Router and vLLM connector cache transfer and fallback have a separate integration gate.