Downloads · 30 days
62
11% of all-time downloads
axiom-of-choice/gemma-3-4b-es-reasoning-gguf
gemma-3-4b-es-reasoning-gguf is a text generation model from axiom-of-choice. Use it when you need the model to write or continue text. The card lists the license as gemma.
GGUF quantizations for llama.cpp, Ollama and LM Studio. Quantized from a local merge of axiom-of-choice/gemma-3-4b-es-reasoning-peft onto google/gemma-3-4b-it-qat-q40-unquantized.
Downloads · 30 days
62
11% of all-time downloads
All-time downloads
574
Public
Repo size
6.5 GB
Likes
1
Public
Click a slice to open those files.
.gguf6.5 GB · 100%
From the Hugging Face model README
GGUF quantizations for llama.cpp, Ollama and LM Studio. Quantized from a local merge of axiom-of-choice/gemma-3-4b-es-reasoning-peft onto google/gemma-3-4b-it-qat-q4_0-unquantized.
Text-only. google/gemma-3-4b-it-qat-q4_0-unquantized is multimodal; this GGUF is not -- the vision
tower was dropped during conversion (convert_hf_to_gguf.py converts only
the text decoder for this architecture; verified with gguf.GGUFReader: 0
vision tensors in the file). Defensible for a Spanish-reasoning text model,
but this repo cannot see images even though the base it was merged onto can
-- use the bf16 merge, or the source adapter on its multimodal base, if you
need that.
Q4_0Q8_0Shipped as Q4_0, not the usual Q4_K_M default. This base is QAT (quantization-aware training) for exactly the q4_0 scheme -- it's in the base model's own name. Measured directly: Q4_K_M silently produced wrong arithmetic and dropped the Spanish <think> reasoning block that this adapter exists to add, on prompts where Q8_0 (near-lossless) and Q4_0 both answered correctly with full reasoning. Not a hypothesis -- verified with llama-cli on the same question across all three quant levels before publishing. If you need a smaller/different quant than these two, test it against <think> presence and correctness before trusting it; K-quants are not verified safe for this model.
Q4_0 is the smallest with negligible quality loss for most use.
Q8_0, where offered, is near-lossless if you have the RAM/VRAM for it.
ollama run hf.co/axiom-of-choice/gemma-3-4b-es-reasoning-gguf:Q4_0
# or
llama-cli -hf axiom-of-choice/gemma-3-4b-es-reasoning-gguf:Q4_0 -p "¿Cuánto es 17 por 24?"
Held-out Spanish GSM8K, 283 items, llama-server build 9670, temp 0, max 2048
tokens, repetition penalty 1.1 over the last 20 tokens. The base is
google/gemma-3-4b-it-qat-q4_0-unquantized converted and quantized to Q4_0
the same way, so the base and Q4_0 rows share engine and quantization.
| accuracy | closed <think> | looped | Spanish confidence | |
|---|---|---|---|---|
base Q4_0 | 35.3% | 0% | 0% | n/a |
this repo Q4_0 | 73.1% | 92.2% | 7.8% | 0.996 |
this repo Q8_0 | 70.3% | 96.1% | 3.5% | 1.000 |
Paired on the same items (McNemar exact, "only A" / "only B" correct):
| A vs B | only A | only B | p |
|---|---|---|---|
base Q4_0 vs Q4_0 | 15 | 122 | < 0.0001 |
base Q4_0 vs Q8_0 | 21 | 120 | < 0.0001 |
Q4_0 vs Q8_0 | 38 | 30 | 0.40 |
MLX adapter vs Q4_0 | 28 | 40 | 0.18 |
MLX adapter vs Q8_0 | 31 | 35 | 0.71 |
Both files reproduce the result of the MLX adapter they were merged from (68.9%
on the same items); neither is distinguishable from it or from the other, so
Q4_0 is not shown to be better than Q8_0. Individual answers still differ
between any two of these on about 68 of 283 items even though the totals agree.
The merge-verification table below is a separate check, on the bf16 merge at n=20, before quantization.
This is checked against the unmerged PEFT adapter's own generations
(results/peft_parity/gemma-3-4b-s5.6-qat-unquant.json), not the original MLX numbers -- the question here is only
"did merging change the model", and MLX-vs-PyTorch divergence is a separate,
already-documented story on the adapter repo's card.
| accuracy | agreement | |
|---|---|---|
| this repo (merged) | 75.0% | 100.0% |
| unmerged adapter | 75.0% |
Paired counts: 0 correct only here, 0 correct only unmerged, out of 20. The merge tracks the unmerged PEFT adapter closely, as it should -- this is the same engine and precision on both sides, so unlike the MLX-vs-PyTorch phase-2 numbers, there is no expected source of divergence here. Treat any drift beyond this as a merge bug, not backend noise.
Same recipe, dataset and evaluation as the adapter --
see axiom-of-choice/gemma-3-4b-es-reasoning-peft.