gemma-4-31b-it-qcache-q4k

gemma-4-31b-it — Q4_K weight cache + auxiliary tensors

Quantized deployment artifacts for google/gemma-4-31b-it.

• qcache-q4_k.v1.gguf — GGML Q4_K tensors for the model's linear projections, stored in a GGUF container keyed by the original checkpoint tensor paths (no metadata KVs). Produced by in-memory quantization of the bf16 checkpoint with a candle-based loader.
• aux-tensors.safetensors — everything the cache does not carry: embeddings, norms, and the full-attention layers' projections kept in bf16. Keeping the global-attention projections unquantized matters: 4-bit error concentrated in those layers collapses long-range retrieval (needle recall degrades beyond ~2x the sliding window with them quantized, and is exact with them in bf16).
• Config and tokenizer files are copied unmodified from the base repository.

Format note: this is not a llama.cpp-compatible GGUF — tensors keep their original checkpoint names and only linear projections are quantized. Load it with a runtime that pairs the cache with the auxiliary safetensors.

Weights are redistributed under the same terms as the base model.