Qwen3.5-27B-GGUF

Qwen3.5-27B | aard-Q3

A 3-bit, 11 GB GGUF of Qwen/Qwen3.5-27B that fits comfortably in 16 GB VRAM with 32K-token context headroom.

Runs in llama.cpp, LM Studio, Ollama, and any GGUF-compatible runtime.

Bottom line. Token distributions are effectively the same between FP16 and aard-Q3. Across 62 reasoning problems (50 GSM8K + 12 hard eval), zero failures were attributable to quantization — every miss either reproduced on the FP16 reference or was a harness truncation we could fix with more tokens.

At 11 GB, this is the smallest published 3-bit Qwen3.5-27B GGUF and the only one with measured quality numbers against FP16. If you have a 16 GB GPU, there's no reason to run a larger 3-bit quant of this model.

The best 3-bit quant of Qwen3.5-27B

Other 3-bit GGUFs of this model are 13.3–13.9 GB and publish no quality evaluations. Bartowski's own card labels their Q3_K_M "Low quality." This is the smallest 3-bit GGUF of Qwen3.5-27B with published quality numbers.

How it stacks up

Published evals

Held-out set: 21 prompts across code, math, reasoning, factual, creative; 150 response tokens per prompt, greedy decoding.

<details> <summary>GSM8K miss breakdown (4 problems)</summary>

Three of four are harness-level (raising the token budget fixes them); the fourth is a known ambiguity in the GSM8K prompt. With the token fix applied, effective accuracy is 48/50 = 96%. No miss was traced to quantization.

</details>

Quick stats

Download & run

huggingface-cli download aaardpark/Qwen3.5-27B-GGUF \
  Qwen3.5-27B-aaardpark-aard-Q3.gguf --local-dir .

llama-cli -m Qwen3.5-27B-aaardpark-aard-Q3.gguf -ngl 99 -c 32768

Notes on use

• Thinking model. Qwen 3.5 is a reasoning model — it emits a <think>…</think> block before the final answer. Budget at least 1024–2048 tokens for responses on non-trivial prompts; 4096 for hard reasoning. On extraction, split on </think> and take the post-block content.
• Runtime requirements. Needs llama.cpp build 8670 or later (Gated Delta Net / linear_attn support). LM Studio / Ollama users: make sure your bundled runtime is current, or load may fail with a generic error.
• Chat template. Embedded in the GGUF — use the default chat format; no manual templating required.

Why this quant

We don't spend bits — we spend better rounding.

Most 3-bit GGUFs labeled "Q3_K_M" mix in Q4_K and Q5_K tensors to protect sensitive layers (attention-V/K, FFN-down), landing around 4 BPW and 13.5 GB. This one keeps 497 of 498 weight tensors at pure Q3_K with only the output projection at Q6_K (~3.07 BPW). The aggressive pack is normally quality-poisonous. Calibration-guided rounding is what keeps it coherent at this bit budget.

Same file format. Runs anywhere Q3_K_M runs. Smaller file, held-out quality.

Outputs

Debugging

Here's a Python function intended to merge two sorted lists. Find and fix the bugs:

def merge(a, b):
    result = []
    i, j = 0, 0
    while i < len(a) and j < len(b):
        if a[i] < b[j]:
            result.append(a[i])
            i += 1
        else:
            result.append(b[j])
            i += 1
    result.extend(a[i:])
    result.extend(b[j:])
    return result

The bug is in the else branch of the while loop. When a[i] >= b[j], the code appends b[j] but increments i instead of j. This causes the wrong pointer to advance, so later iterations re-read b[j] and eventually walk off the end of a.

def merge(a, b):
    result = []
    i, j = 0, 0
    while i < len(a) and j < len(b):
        if a[i] < b[j]:
            result.append(a[i])
            i += 1
        else:
            result.append(b[j])
            j += 1  # was: i += 1
    result.extend(a[i:])
    result.extend(b[j:])
    return result

---

Explain a concept to a non-expert

I'm a backend engineer who's never touched databases before. Explain the difference between a B-tree index and a hash index in one paragraph, focused on when I'd use each.

A B-tree index organizes data in a sorted tree structure, making it the default choice for general queries because it supports range searches (e.g., "find users with IDs between 100 and 200") and sorting (ORDER BY), whereas a hash index uses a hash function to map keys directly to memory addresses, offering O(1) speed for exact equality lookups (e.g., "find user ID 123") but failing completely for range queries or sorting since the data is unordered. Use a B-tree index for most general-purpose queries where you need to filter by ranges or sort results, and switch to a hash index only when you have a high volume of exact equality lookups and never need to perform range queries or sorting on that column.

---

Diagnose a practical problem

My laptop fan spins up to full speed only when I'm on video calls — nothing else. Chrome tabs, code compilation, games, all fine. Only video calls. Give me three likely causes ranked by probability.

1. Hardware Acceleration disabled (most likely). Video-call apps default to software encoding when hardware acceleration is off, forcing the CPU to do work the GPU normally handles. Games and compilation are tuned for GPU; video calls fall back to CPU and spike it. Check Zoom/Teams/Meet settings and your browser for a "Hardware Acceleration" toggle.

2. AI features (Background Blur / Smart Framing). Background blur and face-tracking run CPU-side real-time segmentation on every frame. Games don't touch this code path, so they stay cool. Turn off "Blur," "Virtual Background," "Smart Framing" and re-test.

3. Power profile triggered by camera activation. Some OEM power managers (Dell, HP, Lenovo) escalate to Performance mode when the webcam turns on, regardless of actual load. Check for camera-linked profile rules in your vendor's power utility.

VRAM requirements

Qwen3.5-27B uses Gated Delta Net (linear attention), so the context state is fixed-size per layer rather than growing with sequence length. Context scales further than standard transformers on the same VRAM.

Details

• Base model: Qwen/Qwen3.5-27B
• Method: Calibration-guided per-tensor 3-bit quantization, aard-Q3 pack via modified llama.cpp writer
• Output projection: Q6_K (all other 497 weight tensors at Q3_K)
• Calibration: Chat-formatted prompts across code, math, reasoning, factual, creative

More from aaardpark

• gemma-4-31B-it — 15.3 GB, 96% GSM8K
• Qwen 2.5 72B Instruct GGUF — 35 GB, 88% GSM8K
• Qwen 2.5 32B Instruct GGUF — 15 GB