Qwen3-32B-BA-GGUF

Qwen3-32B-Business-Agentic-GGUF

GGUF quant of Qwen3-32B, calibrated on agentic and business workflows, and also function calling.

Quantized using llama.cpp with a custom imatrix (importance matrix) in attempt to retain high logical reasoning and tool-calling accuracy even at lower bitrates.

Quantization Details

• Quant Types: Q3_K_L (~3.85 BPW, ~16.1 GB)
• Calibration / imatrix: Calculated with high-context business & code logic dataset.
• Perplexity (PPL): ~3.57 (retains core FP16 capabilities with some minor degradation).

---

Usage Instructions

1. Running with llama-server / llama.cpp (ROCm / CUDA)

To enable tool calling and agentic capabilities, make sure to supply the --jinja flag:

llama-server \
  -m Qwen3-32B-BA-Q3_K_L.gguf \
  --jinja \
  -fa \
  -c 16384 \
  -ngl 64 \
  --port 8080

2. Running with Ollama

Create a Modelfile:

FROM ./Qwen3-32B-BA-Q3_K_L.gguf

PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.3
PARAMETER num_ctx 16384

TEMPLATE """{{- if .Messages }}
{{- range .Messages }}
<|im_start|>{{ .Role }}
{{ .Content }}<|im_end|>
{{- end }}
{{- else }}
<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
{{- end }}"""

Build and run:

ollama create qwen3-32b-ba -f Modelfile

Run local model

ollama run qwen3-32b-ba "Привет! Какие задачи ты умеешь решать?"