Downloads · 30 days
238
20% of all-time downloads
keyuan01/Gemma-4-31B-it-MixQ-14G-GGUF
Gemma-4-31B-it-MixQ-14G-GGUF is a image-text-to-text model from keyuan01. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
- Model: gemma-4-31b-mixq-14g.gguf - Multimodal projector: mmproj-gemma-4-31b-f16.gguf
Downloads · 30 days
238
20% of all-time downloads
All-time downloads
1.2K
Public
Repo size
15.5 GB
Likes
4
Public
Click a slice to open those files.
.gguf15.5 GB · 100%
From the Hugging Face model README
gemma-4-31b-mixq-14g.ggufmmproj-gemma-4-31b-f16.gguf14,338,351,232 bytes13659.02 MiB13.34 GiB14.32 GBThis is the smaller mixed-quant target built from the F16 text GGUF using llama.cpp quantization plus imatrix.
It is not a pure uniform quant.
It uses a mixed recipe:
token_embd -> q8_0output -> q8_0attn_q -> q5_kattn_k -> q5_kattn_v -> q4_kattn_output -> q2_kffn_up -> q3_kffn_gate -> q3_kffn_down -> q2_kRecipe file:
mixq-14g-recipe.txtThis release is a GGUF quantized model for llama.cpp.
Quantization family:
GGUFllama.cppimatrix-guided quantizationMix-Quant)This is not a single uniform Q4 or Q3 file.
It is a mixed-precision build where different tensor groups keep different quant types according to sensitivity and size budget.
imatrix)This build uses llama.cpp importance matrix calibration.
Core formula:
I_j = Σ_t x_{t,j}^2
Where:
x_{t,j} is the activation value of channel j for token/sample step tI_j is the accumulated importance score of that channel across calibration textPractical meaning:
imatrix does not use benchmark scores directly.
It estimates sensitivity from activations collected on calibration data.
Yes. Multimodal remains supported when paired with:
mmproj-gemma-4-31b-f16.ggufNotes:
mmproj together with the main modelThe working road was:
Observed tests on this version:
llama.cpp/v1/completions evaluation chain works after cleaning old portsLocal benchmark references:
F16 MMLU-Pro 1400: 0.6421428571Q3 MMLU-Pro 1400: 0.645000000014G MMLU-Pro 200: 0.6150000000Q3 HellaSwag 200: 0.865000000014G HellaSwag 200: 0.8700000000Q3 HellaSwag 1400: 0.8771428571Evaluation files:
evals/mmlu_pro_choose_f16_seed_20260411_n1400.jsonevals/mmlu_pro_choose_q3_seed_20260411_n1400.jsonevals/mmlu_pro_choose_mixq14g_seed_20260411_n200.jsonevals/hellaswag_choose_q3_seed_20260411_n200_new.jsonevals/hellaswag_choose_mixq14g_seed_20260411_n200.jsonevals/hellaswag_choose_q3_seed_20260411_n1400_new.jsonNotes:
MMLU-Pro 140014G vs Q3Tested environment:
llama.cpp built with CUDA supportMinimal setup:
llama.cpp with CUDA.mmproj together if you need vision.llama-server processes before benchmarking.Example text+vision server:
/home/kasm-user/src/llama.cpp-b8756/build/bin/llama-server \
-m 'gemma-4-31b-mixq-14g.gguf' \
--mmproj 'mmproj-gemma-4-31b-f16.gguf' \
-ngl 999 -fa on --ctx-size 4096 -np 1 --port 18081
These datasets were used in the wider tuning and evaluation workflow around this line:
TeichAI/glm-4.7-2000xFarseen0/opus-4.6-reasoning-sft-12kApache note:
Use this version if you want:
mmproj