ainotes-qwen3-4b-q3_k_s

Qwen3-4B Q3_K_S (GGUF)

A 3-bit quantisation of Qwen/Qwen3-4B, cut from the f16 weights published at ggml-org/Qwen3-4B-GGUF using llama-quantize with an importance matrix.

It exists because no 3-bit cut of this model is published, and on a phone the size matters more than the name of the quantisation.

Why this cut

Measured on a Realme RMX2170 (2 big cores at 2.3 GHz, ~3.4 GB usable RAM), extracting action items from five Hinglish and Devanagari voice notes carrying sixteen checkable details, plus two notes carrying no commitments at all:

Same detail recall as the published q4_k_m at 24% less weight, at the cost of about 20 seconds per note. Below three bits the recall drops away.

Two things worth knowing before choosing this over IQ4_XS:

• Smaller is not faster here. Three-bit k-quants pay dequantisation work that two ARM cores do not absorb, so this file is smaller and slower than IQ4_XS. Pick it when memory is the constraint, not when speed is.
• The detail metric carries about ±1 of noise. Q3_K_M, which is larger, scored 14/16 in the same run. Read this as "roughly 15", not as an exact rank.

Importance matrix

Calibrated on 1570 synthetic voice notes covering Hinglish and eight Indian scripts, in a fixed few-shot prompt shape, rather than on English wikitext.

In fairness to wikitext: at q4_k_m this made no measurable difference — a matrix-calibrated cut and the published one both scored 15/16. The matrix is retained here because low-bit cuts are where calibration is generally expected to matter most, not because this benchmark demonstrated that it did.

Intended use

Built for an offline Indian-market voice note-taker: transcript in, JSON action items out, no network. General-purpose use is fine but untested — everything above measures one narrow task.

Attribution

Base model © the Qwen team, Apache-2.0. Quantised with llama.cpp (b6193).