Downloads · 30 days
278
42% of all-time downloads
Kevletesteur/Qwen3.5-35B-A3B-IQ3_S-custom-mix
Qwen3.5-35B-A3B-IQ3_S-custom-mix is a text generation model from Kevletesteur. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
Hand-crafted mixed-precision GGUF with 317 tensor overrides and custom imatrix. Built before the automated RAMP pipeline — this is the manually optimized version that informed RAMP's design.
Downloads · 30 days
278
42% of all-time downloads
All-time downloads
668
Public
Repo size
15.8 GB
Likes
0
Public
Click a slice to open those files.
.gguf15.8 GB · 100%
From the Hugging Face model README
Hand-crafted mixed-precision GGUF with 317 tensor overrides and custom imatrix. Built before the automated RAMP pipeline — this is the manually optimized version that informed RAMP's design.
| Metric | Value |
|---|---|
| File size | 14.71 GB (3.56 BPW) |
| Base quant | IQ3_S with custom imatrix |
| Tensor overrides | 317 rules (Q5_K/Q8_0 critical paths) |
| Gen speed (ik_llama sm120, ncmoe=2) | 98.6 tok/s |
| Gen speed (stock, ncmoe=4, 2 slots) | 72.1 tok/s (solo) / 45.2 tok/s (dual) |
| Prompt processing | 228 tok/s (solo), 1357 tok/s (ik_llama) |
| Context | up to 128K (2 slots) or 196K (3 slots) |
| Tool-calling | 20/20 PASS |
| KV cache sweet spot | q8_0 keys + q4_0 values |
| Custom-mix | RAMP-v2 | |
|---|---|---|
| Size | 14.71 GB | 15.2 GB |
| BPW | 3.56 | 3.78 |
| Method | Manual 317 overrides | Automated RAMP pipeline |
| imatrix | From Q4_K_M proxy | From RAMP-v1 bootstrapped |
| Multi-slot | 2-3 slots stable | 1 slot optimized |
| Speed (ik_llama) | 98.6 tok/s | 90 tok/s |
The custom-mix is faster and smaller but was hand-tuned. RAMP-v2 is more principled (data-free sensitivity analysis) and has higher quality on SSM paths (Q8_0 ssm_out vs Q5_K).
| Tensor category | Quant type | Count | Why |
|---|---|---|---|
| MoE experts (ffn_gate/up/down_exps) | IQ3_S | ~120 | 93% of params, sparse activation |
| Attention + shared experts | Q5_K | ~203 | Critical path, always active |
| SSM alpha/beta/out, attn_v, output | Q8_0 | ~111 | Highest sensitivity |
| Norms, SSM conv1d/dt/a | F32 | ~301 | Small, must be exact |
Built with llama-quantize --imatrix --custom-q and --prune-layers 40 (MTP removed).
# With ik_llama (fastest)
./llama-server -m Qwen3.5-35B-A3B-IQ3_S-custom-mix.gguf \
-ngl 99 --n-cpu-moe 2 -np 1 -c 65536 \
--cache-type-k q8_0 --cache-type-v q4_0
# With stock llama.cpp (multi-slot)
./llama-server -m Qwen3.5-35B-A3B-IQ3_S-custom-mix.gguf \
-ngl 99 --n-cpu-moe 4 -np 2 -c 131072 \
--cache-type-k q8_0 --cache-type-v q4_0
Kevin Remondiere — Independent ML researcher, Oloron-Sainte-Marie, France