Qwen3.6-35B-A3B-TQ3_4S

Qwen3.6-35B-A3B-TQ3_4S

GGUF quantization of Qwen/Qwen3.6-35B-A3B using TQ3_4S with mixed-precision MoE compression — 2-bit experts, 4-bit attention.

Files

Quantization

MoE experts tolerate aggressive compression because only 8/256 are active per token. This quantization exploits that asymmetry:

Runtime Requirement

This model requires the public TurboQuant runtime fork:

• https://github.com/turbo-tan/llama.cpp-tq3

Recommended Settings (16GB VRAM)

./build/bin/llama-server \
  -m Qwen3.6-35B-A3B-TQ3_4S.gguf \
  -ngl 99 -c 4096 -np 1 \
  -ctk q4_0 -ctv tq3_0 -fa on \
  --jinja \
  --reasoning off --reasoning-budget 0 --reasoning-format deepseek

With vision:

./build/bin/llama-server \
  -m Qwen3.6-35B-A3B-TQ3_4S.gguf \
  --mmproj mmproj-BF16.gguf \
  -ngl 99 -c 4096 -np 1 \
  -ctk q4_0 -ctv tq3_0 -fa on \
  --jinja --no-mmproj-offload \
  --reasoning off --reasoning-budget 0 --reasoning-format deepseek

Performance (RTX 5060 Ti 16GB)

Fits entirely in 16GB VRAM — no CPU offload needed.

Quality

10/10 correct on standard QA benchmark (capital of France, 2+2, Python reverse string, gravity, WW2, primes, boiling point, Shakespeare, Jupiter, hello→Hola).

Base Model

• Qwen/Qwen3.6-35B-A3B
• Source: unsloth/Qwen3.6-35B-A3B-GGUF (Q8_0)

License

Apache 2.0 — same as the base model.

Tool Call Validation

Tested with --jinja on both --reasoning off and --reasoning on --reasoning-budget 2048:

Recommended settings for tool-use / agentic workflows

--jinja --reasoning off --reasoning-budget 0 --reasoning-format deepseek

Avoid --presence-penalty above 0.5 for tool-use — high values diversify reasoning tokens but don't improve structured JSON output, and can cause repeated near-identical tool calls in agent loops.

If using --reasoning on, ensure your agent framework detects consecutive identical tool calls and breaks after 2-3 retries.

Run tests yourself

chmod +x test_tool_calls.sh
./test_tool_calls.sh 8085