Downloads · 30 days
79
14% of all-time downloads
keyuan01/Gemma-4-31B-it-MixQ-Q3-16G-GGUF
Gemma-4-31B-it-MixQ-Q3-16G-GGUF is a image-text-to-text model from keyuan01. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
- Model: gemma-4-31b-mixq-q3.gguf - Multimodal projector: mmproj-gemma-4-31b-f16.gguf
Downloads · 30 days
79
14% of all-time downloads
All-time downloads
545
Public
Repo size
18 GB
Likes
1
Public
Click a slice to open those files.
.gguf18 GB · 100%
From the Hugging Face model README
gemma-4-31b-mixq-q3.ggufmmproj-gemma-4-31b-f16.ggufThis is a conservative mixed-quant GGUF build of Gemma 4 31B for llama.cpp.
It is not a pure uniform quant. It was built with:
imatrixThis release is a GGUF quantized model for llama.cpp.
Quantization family:
GGUFllama.cppimatrix-guided quantizationMix-Quant)This means the model is not stored with one single quant type everywhere. Instead, different tensor groups are assigned different precision levels according to sensitivity.
imatrix)This build uses llama.cpp importance matrix calibration.
Core formula:
I_j = Σ_t x_{t,j}^2
Where:
x_{t,j} is the activation value of channel j for token/sample step tI_j is the accumulated importance score of that channel across calibration textPractical meaning:
imatrix does not use benchmark scores directly.
It estimates sensitivity from activations collected on calibration data.
Yes. Multimodal remains supported when used together with:
mmproj-gemma-4-31b-f16.ggufNotes:
mmprojThe practical road was:
Observed checks during the project:
llama.cppmmproj is presentLocal benchmark references:
F16 MMLU-Pro 1400: 0.6421428571Q3 MMLU-Pro 1400: 0.6450000000Q3 HellaSwag 200: 0.8650000000Q3 HellaSwag 1400: 0.8771428571Evaluation files:
evals/mmlu_pro_choose_f16_seed_20260411_n1400.jsonevals/mmlu_pro_choose_q3_seed_20260411_n1400.jsonevals/hellaswag_choose_q3_seed_20260411_n200_new.jsonevals/hellaswag_choose_q3_seed_20260411_n1400_new.jsonNote:
MMLU-Pro 1400This line was built and tested with:
llama.cpp build with CUDA supportMinimal environment steps:
llama.cpp with CUDA enabled.mmproj-gemma-4-31b-f16.gguf next to the main GGUF if vision is needed.-ngl 999 -fa on.Example text+vision server:
/home/kasm-user/src/llama.cpp-b8756/build/bin/llama-server \
-m 'gemma-4-31b-mixq-q3.gguf' \
--mmproj 'mmproj-gemma-4-31b-f16.gguf' \
-ngl 999 -fa on --ctx-size 4096 -np 1 --port 18081
These datasets were used in the broader tuning and testing workflow around this model line:
TeichAI/glm-4.7-2000xFarseen0/opus-4.6-reasoning-sft-12kApache note:
This version is the larger and more conservative mixed-quant line.
Use this version if you want:
mmproj