Downloads · 30 days
170
7% of all-time downloads
nivvis/Qwen3.5-35B-A3B-EQ-v5-GGUF
Qwen3.5-35B-A3B-EQ-v5-GGUF is a text generation model from nivvis. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
GGUF quantizations of nivvis/Qwen3.5-35B-A3B-EQ-v5 for llama.cpp, Ollama, and LM Studio.
Downloads · 30 days
170
7% of all-time downloads
All-time downloads
2.5K
Public
Repo size
90.5 GB
Likes
0
Public
Click a slice to open those files.
.gguf90.5 GB · 100%
From the Hugging Face model README
GGUF quantizations of nivvis/Qwen3.5-35B-A3B-EQ-v5 for llama.cpp, Ollama, and LM Studio.
| Quant | Size | BPW | Notes |
|---|---|---|---|
| F16 | ~65 GB | 16.01 | Full precision, lossless conversion |
| Q4_K_M | ~20 GB | 4.88 | Best 4-bit balance, recommended for most users |
A DPO fine-tune of Qwen3.5-35B-A3B-heretic-v2.
The tune optimized for two things:
This is still intended as a general use model (agentic, coding, general chat). Tuning was lightly & with precision. More general benchmarks to follow.
This model is trained to be a better conversational partner in emotionally complex situations, while maintaining base model capabilities. It:
| Base | Qwen/Qwen3.5-35B-A3B |
| Parent | llmfan46/Qwen3.5-35B-A3B-heretic-v2 (decensored via MPOA+SOMA) |
| Fine-tune | DPO with LoRA (r=32, alpha=64) |
| Training data | DPO preference pairs with diverse, simulated (real-situation-based) generated dialogue |
Ranked #8 on raw score* EQ-Bench 3 with only 3B active parameters — competitive with frontier models at a fraction of the compute.
| # | Model | Raw Score |
|---|---|---|
| 1 | horizon-alpha | 202.3 |
| 2 | Kimi-K2-Instruct | 202.0 |
| 3 | gemini-2.5-pro-preview-06-05 | 200.5 |
| 4 | o3 | 199.0 |
| 5 | gpt-5 | 195.6 |
| 8 | EQ-v5 (this model, 3B active) | 193.6 |
| 10 | claude-opus-4 | 192.6 |
*Table lists all models available in EQ-Bench 3 repo (so known judge, settings etc so we can be as apples to apples on raw score). Still raw score is not ideal. ELO submission pending. Better than no stats!
See the BF16 model card for full benchmarks and training details.
llama-server \
-m Qwen3.5-35B-A3B-EQ-v5-Q4_K_M.gguf \
--host 0.0.0.0 --port 30000 \
-ngl 99 --jinja
Split shards: point to the -00001-of-* file, llama.cpp auto-detects the rest.
ollama run hf.co/nivvis/Qwen3.5-35B-A3B-EQ-v5-GGUF:Q4_K_M
This model supports thinking mode. To disable (for faster, direct responses):
{"chat_template_kwargs": {"enable_thinking": false}}
temp=1.0, top_p=0.95, top_k=20, presence_penalty=1.5temp=0.7, top_p=0.8, max_tokens=2048Qwen/Qwen3.5-35B-A3B
→ llmfan46/Qwen3.5-35B-A3B-heretic-v2 (decensored)
→ nivvis/Qwen3.5-35B-A3B-EQ-v5 (DPO for EQ)
→ this repo (GGUF quantizations)
Apache 2.0, following the base Qwen3.5 license.