Qwen3.6-35B-A3B-MLX-mixed-6.4bit

<div align="center"><img width="400px" src="https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.6/logo.png"></div>

Qwen3.6-35B-A3B

Quality: quantized (mixed quants per tensor, group size: 32, 6.439 bpw)

Quantization: 5 bit for experts, 8 bit for head, shared experts and attention tensors, bf16 for embeddings and some linear attention tensors.

Average bits per weight: 6.439.

Recommended settings

1. Sampling Parameters:
   • The developers suggest using the following sets of sampling parameters depending on the mode and task type:
     • Thinking mode for general tasks:
       temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0
     • Thinking mode for precise coding tasks (e.g., WebDev):
       temperature=0.6, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0
     • Instruct (or non-thinking) mode for general tasks:
       temperature=0.7, top_p=0.8, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0
     • Instruct (or non-thinking) mode for reasoning tasks:
       temperature=1.0, top_p=1.0, top_k=40, min_p=0.0, presence_penalty=2.0, repetition_penalty=1.0
   • For supported frameworks, you can adjust the presence_penalty parameter between 0 and 2 to reduce endless repetitions. However, using a higher value may occasionally result in language mixing and a slight decrease in model performance.

---

Source

This model was converted to MLX format from Qwen/Qwen3.6-35B-A3B using mlx-vlm version 0.4.4.