Qwen3.8-27B-ByteShape-IQ4_XS-ASCII-GGUF

Qwen3.8-27B ByteShape IQ4_XS-3.84bpw — ASCII-P1M vocab

ByteShape's Qwen3.8-27B IQ4_XS-3.84bpw (ShapeLearn per-tensor quantization) with its vocabulary pruned to ASCII plus math/typography symbols, using bsaleh03/ASCII-Condensed-prune-tools (--policy P1M).

English and code only. See Limitations.

What changed

Only token_embd.weight (IQ4_XS, 0.675 → 0.352 GB) and output.weight (Q6_K, 1.043 → 0.543 GB) were touched. Rows are gathered in quantized space: no dequantize/requantize, so every kept row is bit-identical to the source. The tokenizer arrays and merges were rewritten and the special-token ids remapped. The 256 byte-fallback tokens, all specials and all partial-UTF-8 fragments are kept, so any text can still be tokenized.

verify_prune.py --policy P1M passed every check:

• all 864 non-vocab tensors byte-identical to the source
• sampled vocab rows identical to their source rows
• metadata preserved
• kept set exactly what P1M specifies

Tool commit: 376a426d5c6b30f28e03c6efaa9edcf85048773f.

Measurements

Measured on an RTX 5060 Ti 16 GB with llama.cpp v0.4.1.

Perplexity: ctx 4096, 30 chunks, f16 KV. Lower is better, and the figures are deterministic, so they compare exactly across rows.

On ASCII-only text the prune costs nothing (1.6488 vs 1.6490). The +2.9% on the mixed code corpus comes entirely from its 1.27% non-ASCII characters, mostly Bangla string literals.

Context and speed: whole model on GPU, q4_0 KV cache, -ub 512, MTP off.

A 16 GB card with a light desktop fits about 128k of context; 160k needs a nearly idle desktop.

Agentic coding eval (10 executable tasks, thinking on, temp 1.0 / top_p 0.95 / top_k 20): 8/10, 18,381 tokens total. One of the two failures is a grading artifact: the answer ended with a usage-example code block, which the grader took for the solution.

Tool calling and mid-conversation system messages work with a patched chat template (see below).

Usage

llama-server -m Qwen3.8-27B-ByteShape-IQ4_XS-3.84bpw-ASCII-P1M.gguf \
  -c 131072 -ngl 99 -fa on --cache-type-k q4_0 --cache-type-v q4_0 \
  --jinja --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0

• MTP: the head is still in the file; add --spec-type draft-mtp --spec-draft-n-max 2 if you have the VRAM for the draft context.
• Claude Code and similar agents: the stock Qwen3.8 template raises System message must be at the beginning. on mid-conversation system messages. Patch that branch to render them as normal system blocks.

Limitations

• Non-Latin text breaks. Scripts outside ASCII and the P1M symbol set (Bangla, CJK, Arabic, Cyrillic, …) fall back to one token per UTF-8 byte. The model was never trained on those byte sequences, so it reads them as garbled text. A Bangla prompt got a romanized reply saying the input looked garbled; the full-vocab model answers it correctly. For multilingual use, take the source file instead.
• Accented Latin (é, ü, ñ) is not in P1M either, so it also degrades.
• Not affiliated with ByteShape or the Qwen team. All numbers above come from one machine.

Credits

• Base model: Qwen/Qwen3.8-27B (Apache-2.0)
• Quantization: ByteShape (ShapeLearn)
• Vocab pruning: bsaleh03/ASCII-Condensed-prune-tools