Qwen3.8-Flash-Next-IQ6ish_IK

IK-Llama GGUF for https://huggingface.co/Qwen/Qwen3.8-Flash-Next

This GGUF only works with the IK-Llama fork https://github.com/ikawrakow/ik_llama.cpp

This is a custom mixed quantization of Qwen3.8-Flash-Next / Qwen4Exp, produced primarily with ik_llama.cpp and a custom, broad-spectrum importance matrix. The quant mixture was chosen for my specific hardware (72GB VRAM + 128GB system RAM), but if you can find a use for it, go nuts.

• Routed MoE gate and up expert tensors: IQ5_K_R4
• Routed MoE down expert tensors: Q5_0
• PLE per_layer_token_embd.weight: Q8_0
• Token embeddings and output tensor: Q8_0
• Shared experts and most remaining quantizable BF16 weights: Q8_0
• F32 norms, routers, biases, and similar tensors: retained as F32
• blk.1.ple_conv1d.weight: retained as BF16

Total: 1,224 tensors
Model size: ~133.08 GiB
Average: 6.461 BPW

The very large PLE tensor was incorporated from an upstream-generated Q8_0 shard because directly requantizing that tensor with ik_llama exceeded available memory. The remaining model shards were quantized from BF16 using ik_llama and the custom importance matrix.

##Importance Matrix Dataset

The importance matrix was generated from a custom ~1.82M-token general-purpose calibration corpus using the Qwen3.8-Flash-Next tokenizer.