Downloads · 30 days
48
8% of all-time downloads
peasantsmith/Qwen3.5-122B-A10B-PS-IQ2_XXS-MTP-GGUF
Qwen3.5-122B-A10B-PS-IQ2_XXS-MTP-GGUF is a machine learning model from peasantsmith. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
The first IQ2XXS-class GGUF of Qwen3.5-122B-A10B — smaller than any published quant (unsloth's ladder floors at Q3KS 52.5 GB) — built entirely from the original Qwen/Qwen3.5-122B-A10B weights with the native MTP (next…
Downloads · 30 days
48
8% of all-time downloads
All-time downloads
567
Public
Repo size
41.8 GB
Likes
3
Public
Click a slice to open those files.
.gguf41.8 GB · 100%
From the Hugging Face model README
The first IQ2_XXS-class GGUF of Qwen3.5-122B-A10B — smaller than any published quant (unsloth's ladder floors at Q3_K_S 52.5 GB) — built entirely from the original Qwen/Qwen3.5-122B-A10B weights with the native MTP (nextn) head embedded at Q8_0.
File: Qwen3.5-122B-A10B-PS-IQ2_XXS-MTP.gguf — 41.77 GB (38.9 GiB), ≈2.7 bits/param over 122B total params.
SHA-256: 01bd59623e7d4f2f45bd1ce8665042bf449b8000a76480281bef7866cfce5557
Qwen/Qwen3.5-122B-A10B official BF16 safetensors (39 shards, 244 GB — no third-party quant or modified repo involved)convert_hf_to_gguf.py (arch qwen35moe, nextn_predict_layers = 1, MTP remapped to blk.48.* and embedded at full precision)llama-quantize importance-matrix quant with the tensor split belowImatrix: imatrix_unsloth.gguf from unsloth/Qwen3.5-122B-A10B-GGUF (359 MB, large diverse calibration corpus). Verified before use: 144/144 main expert tensors covered, zero naming mismatches against our master; the only uncovered tensors are the 3 MTP expert tensors, which were pinned out of IQ quantization.
| Class | Tensors | Type |
|---|---|---|
| Routed expert gate/up | 96 | IQ2_XXS |
| Routed expert down | 48 | Q2_K |
| Token embeddings | 1 | Q6_K |
| Attention, shared experts, router, output, norms | 384 | Q8_0 |
| Layer norms/biases | 370 | F32 (kept) |
| MTP / nextn block (blk.48) | 20 | Q8_0 |
Down-projection gets one full step above gate/up because its errors land directly and additively on the residual stream, while gate/up errors pass through activations first. The MTP head is kept at maximum fidelity so the draft head stays sharp.
| Test | Result |
|---|---|
| Greedy smoke ("The capital of France is") | ✅ "The capital of France is Paris." (finish=stop) |
| Needle recall ("9137 steps", 8k ctx) | ✅ exact answer 9137 |
| LRU cache code task (8k ctx) | ✅ complete correct doubly-linked-list LRU with sentinel nodes, O(1) get/put, eviction (5000-token continuous generation, zero degradation) |
| Long-generation stability | ✅ 5000 tokens continuous at steady t/s |
| Perplexity (wikitext-2) | not measured — see honesty note below |
llama-server -m Qwen3.5-122B-A10B-PS-IQ2_XXS-MTP.gguf -ngl 99 -ncmoe 20 -c 8192 --load-mode none
Requires a recent llama.cpp master (qwen35moe + linear-attention + MRoPE support). The model ships its chat template; use --jinja.
-ncmoe-style expert pinning is llama.cpp-onlyollama derives its tag from the GGUF filename (last dash-separated segment):
ollama run hf.co/peasantsmith/Qwen3.5-122B-A10B-PS-IQ2_XXS-MTP-GGUF:IQ2_XXS
Notes:
qwen35moe architecture yet); the model runs as a standard A10B MoE.-ncmoe are llama.cpp-only; ollama loads with its default placement. Measured ~17 t/s with llama.cpp on 3x RTX 3060 (experts partially on CPU) - see benchmarks below.Upstream model by Qwen (Apache-2.0). imatrix by Unsloth. Quant by Peasant Smith.